First, I considered posting this on the academia SE, but since this is more a question of statistical validity, I thought it more appropriate to post here. If there are similar questions/answers here or there, please direct me to them, as I've searched for a while and not yet found anything useful.
A colleague with the same background as me (Ecology) is now working in a different research field and has started a project. Both of us have concerns about the validity of tests and results used in similar studies (potential pseudo-replication, wrong choice of hypothesis test, not correcting for multiple comparisons), yet it seems pervasive in the field. Relevant journals have decent impact factors, and I assume that this field has resolved any issues that we perceive as problematic.
I recognize there are often differences in how data or results are presented from one discipline to another, and I may simply be overlooking something that is implied or established knowledge in a field different from mine. However, the 'Statistical Analysis' sub-heading within Methods often does not describe any actual methods; rather, it states (in a single sentence or two) the software used, and relegates the actual hypothesis test used and associated stats/metrics to Figure captions. This baffles me.
Journals typically require detailed statistical analysis methodology explaining the reasoning for choosing a particular test, that assumptions were checked and data transformed if necessary, type of post-hoc analysis, etc. However, I am encountering articles that lack this detail and present figures that (to me) look like results of ANOVA, but state that t-tests were used. Thus, I'd like to ask some questions here in hopes that the community can help me understand if/why these particular analyses are correct.
It is not my goal to find errors or call out publications, just to understand the analyses and address any gaps in my own knowledge. My question is:
Am I allowed to refer to a particular article(s) with the analysis I have a question about, or do I need to create a similar reproducible example with relevant context?