This is a known problem in epistemology, the problem of infinite regress
The regress problem (also known as Agrippa's Trilemma) is the problem
of providing a complete logical foundation for human knowledge. The
traditional way of supporting a rational argument is to appeal to
other rational arguments, typically using chains of reason and rules
of logic. A classic example that goes back to Aristotle is deducing
that Socrates is mortal. We have a logical rule that says All humans
are mortal and an assertion that Socrates is human and we deduce that
Socrates is mortal. In this example how do we know that Socrates is
human? Presumably we apply other rules such as: All born from human
females are human. Which then leaves open the question how do we know
that all born from humans are human? This is the regress problem: how
can we eventually terminate a logical argument with some statements
that do not require further justification but can still be considered
rational and justified?
There are various approaches to address this, not necessarily mutually exclusive:
Foundationalism
Foundationalists respond to the regress problem by asserting that
certain "foundations" or "basic beliefs" support other beliefs but do
not themselves require justification from other beliefs. These beliefs
might be justified because they are self-evident, infallible, or
derive from reliable cognitive mechanisms. Perception, memory, and a
priori intuition are often considered possible examples of basic
beliefs.
Coherentism
Another response to the regress problem is coherentism, which is the
rejection of the assumption that the regress proceeds according to a
pattern of linear justification. To avoid the charge of circularity,
coherentists hold that an individual belief is justified circularly by
the way it fits together (coheres) with the rest of the belief system
of which it is a part. This theory has the advantage of avoiding the
infinite regress without claiming special, possibly arbitrary status
for some particular class of beliefs. Yet, since a system can be
coherent while also being wrong, coherentists face the difficulty of
ensuring that the whole system corresponds to reality. Additionally,
most logicians agree that any argument that is circular is, at best,
only trivially valid. That is, to be illuminating, arguments must
operate with information from multiple premises, not simply conclude
by reiterating a premise.
Infinitism
An alternative resolution to the regress problem is known as
"infinitism". Infinitists take the infinite series to be merely
potential, in the sense that an individual may have indefinitely many
reasons available to them, without having consciously thought through
all of these reasons when the need arises. This position is motivated
in part by the desire to avoid what is seen as the arbitrariness and
circularity of its chief competitors, foundationalism and coherentism.
The most prominent defense of infinitism has been given by Peter Klein.
And combinations of the above:
Foundherentism
An intermediate position, known as "foundherentism", is advanced by
Susan Haack. Foundherentism is meant to unify foundationalism and
coherentism. Haack explains the view by using a crossword puzzle as an
analogy. Whereas, for example, infinitists regard the regress of
reasons as taking the form of a single line that continues
indefinitely, Haack has argued that chains of properly justified
beliefs look more like a crossword puzzle, with various different
lines mutually supporting each other.[70] Thus, Haack's view leaves
room for both chains of beliefs that are "vertical" (terminating in
foundational beliefs) and chains that are "horizontal" (deriving their
justification from coherence with beliefs that are also members of
foundationalist chains of belief).
Personally I am of the opinion that a combination of the solutions (eg Foundherentism) might be the most appropriate. It may be the case that a set of epistemic facts can not be further analysed and have to be taken as simply given, and another set of facts (or even the same set) refer to each other at various levels, producing a coherent whole, supporting each other.