Embedded Files
Online analysis decision advice
This page prompts you to think through relevant questions that help plan a review and decide what analysis methods to use. The questions are likely to be relevant to almost all reviews to help guide your decision process. However, beware that the final advice you gain here is offered only as a quick opinion, it might not always give you the perfect answer and you might want to consult with, or recruite to your team, a space medicine systematic review methods expert. Remember that methods need to be tailored to each question and it might not be appropriate or safe to repeat the same methods for every review question. The guidance here is simplified to quickly help you think through relevant analysis questions, you may want to read the handbook on the guides page and/or the Cochrane handbook for more detailed advice.
q1- Are there controlled trials with independent groups?
Meta analysis assumes multiple controled trials investigating the same measures with each trial having two independent groups. You will need to extract mean, standard deviation and sample size of each group. The outcomes also need to similar enough to make pooling safe. You can read more in the Cochrane handbook
Yes
You can probably conduct a quantitative meta analysis. You can use off the shelf Cochranes methods and should download RevMan. You could still consider combining a qualitative analysis.
No
It might not be safe to meta anlyse the data. You should report the reason why (no controlled trials, high heterogeniety of outcome measures etc.). You can also consider a qualitative analysis.
Continue to the next question
q2- Are there disparate controlled trials or within participant studies
If you have controlled trials but the outcome measures are too heterogenious to safely pool in meta-analysis, or you have within participant studies such as before and after designs, you can likely perform an effect size analysis. This analysis converts all the data from the included studies to standardised units so you can plot all the data on a single plot and get an overview of the evidence base. You still need to be able to extract mean, standard deviation and sample size data to run this analysis. This is a conservative method that avoids violating statistical assumptions.
Yes
You can probably conduct a quantitative effect size analysis. You can use the data extraction and analysis sheet to complete the analysis. You can still consider combining with a qualitative analysis
No
It might not be safe to perform a quantitative analysis on the data you have available. You should report the reason why (lack of published data etc.). You can also consider if a qualitative question might be more appropriate.
Continue to the next question.
q3- Is there data to enable significance reporting?
Think carefully about this as it is a lower quality analysis. If you have articles that indicate a significant finding but with no way of extracting data to convert to standardised units, then you can report the significant results. You can count how many significant reports favour something vs finding it trivial or favouring the controls.
Yes
You can consider if it would be beneficial to your review to include some significance reporting. Use this analysis with caution. You could still consider combining a qualitative analysis.
No
There may not be quantitative data to analyse. You may want to consdier broadening the scope of your review. You can also consider a qualitative analysis or publishing an empty review.
Continue to the next question
q4- Is there any quantitative data available?
If you have answered no to Q1, Q2 and Q3 then there may not be any available data to perform a quantitative analysis. You can consider broadening the scope of your review, changing your question to look at qualitative aspects, or publishing an empty review to clearly and authoritatively demonstrate a research gap.
Yes
Revisit Q1, 2 and 3 to see what analysis you could do, if none are appriate than see the answer for no to this question. You can also consult the Cochrane handbook section about when meta analysis can not be undertaken for in depth methods insights.
No
Consider if you can still provide useful insights from conducting a qualitative analysis. You can also publish an empty review. This is where you publish your search strategy and show definitively that there is no data availble at the current time. This is a strong way to demonstrate a gap in the evidence base.
q5- Would a qualitative question add useful insights?
Qualitative questions try establish the nature of a phenomanon. They tend to ask why, what, when, who type questions rather than asking about the effect of an intervention. An example of a qualitative question is "what technical constraints exist that affect the capability of astronauts to exercise during spaceflight?". You can always consider performing a qualitative analysis with a quantitative one in an integrated or 'mixed methods' approach. This approach can give a more holistic answer and overview of the evidence base. Or you can do a qualitative analysis instead of quantitative should data be lacking.
Yes
Consider doing a qualitative analysis. You should read the qualitative guide to help you understand what this involves. There is also a guide for how to embed thematic analysis. You might also want to consider broadening your search to look at more grey literature such as technical reports from sources such as the NASA technical reports server.
No
If you are not sure about qualitative analysis, consider asking a qualitative expert to help.
If no quantitative or qualitative exists on your topic, then you will need to either broaden your scope, change question or publish an empty review. This is where you publish your search strategy and show definitively that there is no data availble at the current time. This is a strong way to demonstrate a gap in the evidence base.
q6- Can you expand your scope
It is a common problem in space medicine that the perfect data does not exist. Usually review teams are interested in astronaut health/space medicine operations. However, there are few controlled trials with astronaut data and sample sizes tend to be very small compared to other medical fields. For this reason reviews often broaden the scope to include research participants from ground based spaceflight simulation studies. This could include bed rest, immersion, suspension, parabolic flight or isolation environments. You could consider expanding your scope to these populations or looking at different outcome measures or interventions.
Yes
Consider expanding your scope. However, be careful that you will need to address the transferability of broader populations. Not all outcome measures may transfer data well from simulation environments to actual astonauts. You can use the transferability guides to help, but not all simulations have been assessed for transferability yet. Remember it is okay to publish an empty review rather than obscure your question or trying to rely on evidence that has poor transferability and could result in unsafe findings.
No
If no quantitative or qualitative exists on your topic, then you can still publish an empty review. This is where you publish your search strategy and show definitively that there is no data availble at the current time. This is a strong way to demonstrate a gap in the evidence base.
Google Sites
Report abuse