Before we decide if something is true or not (e.g. "is there a God?"), we need to have some kind of process for deciding. So what is that process? I'll quote Steven Pinker:
In reality, science... looks more like Bayesian reasoning.... A theory is granted a prior degree of credence, based on its consistency with everything else we know. That level of credence is then incremented or decremented according to how likely an empirical observation would be if the theory is true, compared with how likely it would be if the theory is false.
You could visualize Bayesian reasoning by imagining a set of balance scales. As an example, say the left side represents the theory "it rained recently" while the right side represents "it didn't rain recently." The scale's initial weighting represents our prior degree of credence: if we're in Utah, then the right side of the scale would be weighted quite heavily, but it would be weighted less heavily if we're in Malaysia.
Now suppose we make an observation: the sidewalk is wet. If it rained recently, then this would be an incredibly likely observation to make. The observation would be less likely if it didn't rain recently. So, we'll add some weight to the left side of the scale.
But how much weight do we add? It depends on two things:
In Utah, sprinklers are a great explanation for wet sidewalks. In addition, rain is pretty rare. So I would put only a little extra weight on the left side of the scale—the right side would still be weighted more heavily, because even given that the sidewalk is wet, I still think it's more likely that it didn't rain recently.
Things would be different in Malaysia. I never saw a sprinkler while I lived there, and it rains pretty much every day. I would put a lot of weight on the left side of the scale, definitely enough that it would outweigh the right side.
This is the same way I think about the existence of God. My prior knowledge consists of all the secular evidence for or against the existence of God (or at least, the portion of that evidence that I've come across). By "secular evidence," I mean roughly anything that might be permissible in an academic journal. This includes historical information, analyses of the textual content of scriptures, evidence about evolution, etc.
Besides secular evidence, I have the evidence of personal experiences. Although these experiences are hard to replicate and thus can't be published in a journal, they're still empirical and thus should be considered. There are many written accounts of what these experiences sometimes feel like. Luke 24:32:
... Did not our heart burn within us, while he talked with us by the way, and while he opened to us the scriptures?
Joseph Smith—History 1:11-12:
11. ... I was one day reading the Epistle of James, first chapter and fifth verse....
12. Never did any passage of scripture come with more power to the heart of man than this did at this time to mine. It seemed to enter with great force into every feeling of my heart. I reflected on it again and again....
Russell M. Nelson, describing an experience performing surgery:
In preparing for that fateful day, I prayed over and over again, but still did not know what to do for his leaking tricuspid valve. Even as the operation commenced, my assistant asked, "What are you going to do for that?"
I said, "I do not know."
We began the operation. ... We found [the valve] to be intact but so badly dilated that it could no longer function as it should. While examining this valve, a message was distinctly impressed upon my mind: Reduce the circumference of the ring. ...
But how? ... Then a picture came vividly to my mind, showing how stitches could be placed... to accomplish the desired objective. I still remember that mental image—complete with dotted lines where sutures should be placed. The repair was completed as diagrammed in my mind. We tested the valve and found the leak to be reduced remarkably. 
Doctrine and Covenants 6:22-23:
22. ... cast your mind upon the night that you cried unto me in your heart, that you might know concerning the truth of these things.
23. Did I not speak peace to your mind concerning the matter? What greater witness can you have than from God?
There are both natural and supernatural explanations for these observations. They could be direct communication from God, or they could be the result of ordinary biological processes (i.e. "it's all in your head"). Each person who has such an experience has to estimate what the likelihood of that experience occurring would be if God exists and what the likelihood would be if God doesn't exist.
So, let's say Alice has had one or more of these personal experiences, and she's convinced that they would be very likely to happen if God exists and very unlikely to happen if God didn't exist. So does that mean, if she uses Bayesian reasoning, that she believes God exists? Not necessarily. It depends on her prior knowledge.
Let me illustrate with another example from Bayesian statistics. This example is so commonly used that I'll just quote it from the first source that came up on Google:
Suppose that you are worried that you might have a rare disease. You decide to get tested, and suppose that the testing methods for this disease are correct 99 percent of the time (in other words, if you have the disease, it shows that you do with 99 percent probability, and if you don't have the disease, it shows that you do not with 99 percent probability). Suppose this disease is actually quite rare, occurring randomly in the general population in only one of every 10,000 people.
If your test results come back positive, what are your chances that you actually have the disease?
Do you think it is approximately: (a) .99, (b) .90, (c) .10, or (d) .01?
Surprisingly, the answer is (d), less than 1 percent chance that you have the disease!
... The basic reason we get such a surprising result is because the disease is so rare that the number of false positives greatly outnumbers the people who truly have the disease. 
In this example, our prior knowledge is that only 1 in 10,000 people has the disease. The observation at hand is that our test came up positive. In other words, our current observation says that there's a 99% chance we have the disease, but our prior knowledge says there's a 99.99% chance we don't have the disease. The test results are overpowered by the strength of our prior knowledge. On the other hand, if 50% of the general population had this disease, then a positive test result would be strong evidence that we have the disease.
Back to Alice. Her personal experiences are like the medical test. They may be quite strong, but they could still be overpowered by her prior knowledge. If she finds the secular evidence to be quite in favor of God not existing, her personal experiences may not be enough to sway her mind—and this is perfectly rational.
So, if you're like Alice and you're trying to decide if you still believe in God or not, trying to seek stronger personal evidence may be a dead end. Instead, you may be better off investigating the secular evidence more deeply. You might find that it's less biased than you first assumed.
The previous section covers methodology. What I've tried to explain is that even in the context of spiritual matters, I attempt to use standard Bayesian reasoning—a rational methodology. So what is faith, and where does it come in?
First, we should be clear that faith is a complex subject, and the scriptures use the term in multiple ways. But in the context of deciding whether or not we believe in God at all, I think of faith as rational thinking in the face of uncertainty, or in other words, a willingness to accept risk.
Humans are often risk-averse and afraid of uncertainty. However, if you make only choices that seem low-risk, you're likely to end up with a suboptimal outcome in the long-run. For many, choosing to marry someone is a decision that comes inseparably bundled with a high amount of uncertainty and risk—but that doesn't imply the right choice is to simply not get married. You have to learn to act rationally despite the uncertainty.
It's the same with faith. Let's consider Bob, the inverse of Alice. His prior knowledge is such that he's willing to consider the existence of God, although alone it probably isn't enough to make him confident that God does in fact exist. Since he hasn't had any personal/spiritual experiences yet, his best bet would be to try validating some claims about how to have these experiences. For example, Moroni 10:4:
And when ye shall receive these things, I would exhort you that ye would ask God, the Eternal Father, in the name of Christ, if these things are not true; and if ye shall ask with a sincere heart, with real intent, having faith in Christ, he will manifest the truth of it unto you, by the power of the Holy Ghost.
And John 7:17:
If any man will do his will, he shall know of the doctrine, whether it be of God, or whether I speak of myself.
Both these claims state that spiritual experiences have prerequisites; they don't just happen. The claims can't be tested without fulfilling the prerequisites, and often those prerequisites can't be fulfilled without significant personal sacrifice. If Bob has an irrational fear of uncertainty—or in other words, a lack of faith—he would probably be unwilling to try testing the claims at all. Even if God does exist, Bob might never know it.
Let's say Bob does test the claims, and he has some spiritual experiences. The evidence may be strong enough that, rationally, Bob should presume God exists and thus live his life accordingly. But even with these spiritual experiences, Bob probably won't be 100% confident. If Bob lacks faith, being less than 100% confident might make him too uncomfortable. He might then irrationally decide to ignore the evidence and continue with the assumption that God doesn't exist.
Faith and reason
Many people believe that faith and reason are separate. Interestingly, both believers and nonbelievers hold this position. Of course, they have different implications. The nonbelievers who hold this position think that because faith and reason are separate, you should throw faith out. The believers who hold this position think you should instead compartmentalize faith and reason—reason should be used in science, and faith should be used in religion.
I think both these positions are off, though. Faith is (or should be) a subset of reason. The same general method of epistemology can be used in both science and religion. We don't have to disagree about methodology.
Importantly, Alice and Bob could end up with different conclusions, even if they both use the same methodology. They've been exposed to different subsets of the available secular evidence, and of course they've had different personal experiences. But let's recognize that that's why they disagree—different sets of evidence and different interpretations of it, not irrationality or faithlessness.
 Enlightenment Now by Steven Pinker
 Su, Francis E., et al. "Medical Tests and Bayes' Theorem." Math Fun Facts. http://www.math.hmc.edu/funfacts.