I think the note you reference by Elliott Sober is not entirely correct. To understand these examples it is helpful to make use of a common distinction between the synchronic and diachronic use of conditional probabilities and of Bayes' theorem.
Bayes' theorem itself is neutral as to the interpretation of probability, i.e. it holds under the Bayesian or frequentist interpretation, or any other. Understood synchronically, i.e. at a particular time or information state, it simply states a relation between two conditional probabilities. The use of the theorem only becomes distinctively Bayesian when it is used diachronically as an updating rule, i.e. when the conditional probability P(B|A) is understood to mean that if I learn that A is true, I should update my degree of belief in B from P(B) to P(B|A).
This distinction is important. Understood synchronically, modus ponens and modus tollens are both probabilistically valid. Ernest Adams in his books, "The Logic of Conditionals", and "A Primer on Probability Logic" showed many years ago that the criterion for the probabilistic validity of an argument is that the improbability of the conclusion cannot exceed the sum of the improbabilities of the premises (where the improbability of A is P(¬A)). In simple cases where we have a few premises and all the probabilities are either close to one or close to zero, this has the consequence that if the premises are highly probable then the conclusion is highly probable.
So, for a given conditional "if A then B", modus ponens is valid, since if P(B|A) is high, and P(A) is high, this entails that P(B) is high. Modus tollens is also valid, since if P(B|A) is high, and P(B) is low, this entails that P(A) is low. Note the synchronic use here: I am not speaking of what happens when we learn something to be true, but what probabilities hold at a particular time or under a particular state of information.
As a slight aside, several other familiar rules from classical logic do not hold in probability logic. In particular, contraposition, hypothetical syllogism, or-to-if, and strengthening of the antecedent do not hold. These are not guaranteed to preserve high probability from premises to conclusion.
When we come to the diachronic use of probabilities, things are different. Strictly speaking, we are no longer doing simple probability theory but adopting a kind of Bayesian epistemological theory about belief revision. Bayesian epistemology is a contentious theory and works only approximately at best. Diachronically, modus tollens may fail. As Sober shows in his example, if P(B|A) is high and I learn that B is false, this does not entail that A is false. I may have just been unlucky. But also with modus ponens, if P(B|A) is high and I learn that A is true, I may be unlucky and B may turn out false.
It should perhaps not come as a surprise that modus ponens can fail in cases of belief revision, since this can happen even in non-probabilistic examples. An example much discussed several years ago runs as follows: "If President Reagan is a Soviet spy, we'll never find out. President Reagan is a Soviet spy. Therefore, we'll never find out." Diachronically, this doesn't work. As soon as we learn the second premise, the conclusion is ruled out.
The moral is that modelling belief revision is much more tricky than working with static probabilities. Learning some proposition to be true requires that we accommodate it among our existing beliefs, and in general there is no simple rule for doing this. This is also one of the reasons why the logic of conditionals is much more complex than that of other logical connectives.
Significance testing is somewhat different, because that is a frequentist method. A frequentist may form a null hypothesis, acquire data that is highly improbable on the basis of that hypothesis, i.e. a low p value, and then at some chosen threshold reject the hypothesis. A Bayesian prefers to take a prior probability for the hypothesis and update on the evidence. The result is not always the same, though in both cases a hypothesis can be disconfirmed by highly improbable evidence.
Using Bayes' theorem in a comparative form to compare two hypotheses is very common. It allows us to eliminate the term P(E) which is often unknown. In the case of comparing naturalistic and theistic theories of some event, it is of little use. The problem with theistic theories is that the probabilities are impossible to assign because they are imponderable. Also, the probabilities are not the only issue. Theistic theories lack the desirable properties of a good theory, such as simplicity, parsimony, explanatory value, and lack of adhocness.
