About Us:
DevTech Systems, Inc. (DevTech) is an international consulting firm dedicated to development, with 40 years of experience providing advisory services and technical assistance to government, private sector, and civil society stakeholders in more than 100 countries. DevTech core practice areas include: Monitoring Evaluation, Research and Learning; Data Solutions; Public Financial Management and Fiscal Sustainability; and Education, Gender, and Youth.
At DevTech, we care deeply about doing work that leads to positive change in the world. We value diverse perspectives and are committed to an inclusive work environment. We encourage each person to learn, develop, and meet their professional potential. We rethink and evolve how we do things to grow and improve our company.
Overview:
Under the Education Technical Support Services contract (ETSS), DevTech is seeking for a Meta Analysis Research Assistant to retrieve information to conduct two systematic reviews of questions #2 and #12 of the USAID Latin America and the Caribbean Education Learning Agenda (see Annex for full activity background)
Q2. How education influences irregular migration. Build on existing systematic review ensuring all databases or gray literature is synthesized.
Q12. What workforce development models after secondary education or higher result in youth employability. Full systematic review required.
Responsibilities:
The Meta Analysis Research Assistant - will support the local research team conducting research through their US college library search engine. This will ensure full access to academic and scientific articles in both the US and LAC.
Scope of Work:
ACTIVITY BACKGROUND:
USAID’s Bureau for Latin America and the Caribbean (LAC Bureau) commissioned the LAC Education Learning Agenda through the USAID-University of Notre Dame (UND) Supporting Holistic and Actionable Research in Education (SHARE) Cooperative Agreement, with the goal of forming questions to help the USAID LAC Bureau, other donors, and implementing partners better understand the best way to support both education recovery from COVID-19 and inclusive education for marginalized populations such as those from 8 lower socioeconomic classes, migrants, Indigenous children and youth, learners with disabilities, those from rural areas, and those from other minority groups. The timing for the development of the learning agenda also coincided with a strong U.S. government focus on addressing the root causes of irregular migration and violence in LAC through early childhood education/ development and youth and workforce development.
To create the learning agenda, UND engaged with a key working group of USAID staff from Washington and USAID’s LAC Missions to define the parameters of the agenda and the process. The process included wide consultations with the LAC Education Learning Hub; key informants from implementing partners, Ministries of Education, and USAID.
As of 2023, the LAC countries and subregions with USAID education programming (bilateral and/or regional) are the Dominican Republic, 11 countries in the Eastern and Southern Caribbean, Ecuador, El Salvador, Guatemala, Haiti, Honduras, Colombia, and Paraguay. The Learning Agenda will contribute to programming in these countries and inform the priorities of future activities and interventions.
The purpose of the USAID LAC Education Learning Agenda is to define institutional learning priorities across the region, linked to programming, decision-making and collaboration efforts. The learning agenda will help USAID to prioritize its primary research, secondary research syntheses, and evidence dissemination efforts until at least 2028. USAID also plans to use the evidence gathered and synthesized from the agenda to guide programming and hopes other donors and implementing partners will also do so.
The final learning agenda has 12 questions and is available here. In addition to developing the learning agenda, the UND team also conducted a quick and high-level rapid evidence stock-taking to determine the level of existing evidence already available to answer each of the 12 learning agenda questions. They used this stock-taking to classify according to the following framework:
They found that none of the questions were at the point of dissemination. They also used the stock-taking to provide a high-level overview of findings from existing evidence for each question (see document for high-level details and the rapid evidence stock-taking document for a list of studies referenced and their key findings).
USAID would like to expand on the rapid-evidence stock-taking that the University of Notre Dame did to conduct a full systematic review and meta-analysis of findings related to each of the learning agenda questions, following the guidance from the Cochrane Collaboration on rigorous systematic reviews. This SOW is focused on Q2 and Q12 as the most pressing to get answers to through consultations with USAID’s LAC Missions and the LAC Education Learning Hub.
The USAID LAC Education Team, USAID Missions, USAID IPI/EDU, LAC Ministries of Education, USAID implementing partners, and other donors will use the systematic review findings to inform the design and implementation of future education interventions in the LAC region. These actors, and especially USAID, will also use the study to inform decisions about priority research questions, topics, contexts moving forward to round out the evidence base. As such, the systematic review should include both findings related to the questions as well as details about what major gaps exist in the research with regards to findings, methods, contexts/locations, etc.
SYSTEMATIC REVIEW METHODS:
USAID seeks two separate rigorous systematic reviews and meta-analyses (to the extent that sufficient studies exist with similar outcomes to inform such meta-analyses; please note that more than one meta-analysis may be needed per systematic review, depending on the number of outcomes identified; DevTech should prioritize the top three or fewer from each systematic review) – one to answer each of the learning agenda questions. The reviews should follow the guidance from the Cochrane Collaborative, as linked above. For the migration question, please note that this systematic review is already advanced and just needs a few researchers to check if there are any untapped databases or gray literature that can be included and then to synthesize the results of the study. Please find, here, a set a documents to help get this review finalized:
- LAC Learning Agenda Implementation Plan – Please see the tab on Existing Studies – where you will find a list of already-pulled studies for all learning agenda questions, including more than 100 studies related to learning agenda question 2. Note, that each of these studies also has details about methods, findings, limitations, etc. These studies should be used to synthesize findings and to determine if any meta-analyses are possible.
- LAC Education and Migration Inclusion Criteria – Please see this document for details on the systematic review process and inclusion/exclusion criteria. DevTech can use this document to determine if there are any untapped resources, including gray literature, that should be consulted to ensure a full and complete body of evidence.
- A short, high-level, summary of the evidence and findings that the researchers who did the data collection for this study think should be included in study.
- The rapid evidence stock-taking document. Please also make sure that all of the studies from this review are included in both reviews.
For the second question, on workforce development, USAID is requesting a full systematic review from start to finish, noting that some documents are available in the implementation plan and rapid evidence stock-taking documents referenced above. All studies included should be checked against a rigorous set of quality criteria to ensure their inclusion is warranted.
The task order is divided into two phases. Phase 1 will focus on conducting a systematic review of the literature, assessing the risk of bias, and determining if a full meta-analysis is possible. If Phase 2 is possible, it would focus on completing the meta-analysis and developing a full report for each RQ as follows:
| # | Deliverable | Description |
Phase 1 | 1 | n/a | Review background documents. Gain familiarity with USAID Learning Agenda and available information on RQ 2 and 12 on Migration and Workforce Development. Review of the preliminary report that Notre Dame developed on RQ2- Migration. |
3 | Study design/protocol. | Approve Study Protocol. Review study methodology and provide guidelines on the use of Cochrane checklist |
4 | Risk of bias tool validation. | Risk of bias. Create/Adapt tool that makes sense. Provide training if needed. |
5 | Final list of titles and abstracts. | QA the study inclusion lists after screening is completed. Advise the team on identifying linked papers. |
6 | Data extraction tool validation. | Data extraction and effect size validation. Review data extraction tool and provide training. Provide effect size definitions, if ES is not available, determine if proxies can be created with available data. |
7 | Preliminary report | Final list of studies that qualify based on the bias rating, risk of bias tool explanation and assessment, and recommendations for or against Meta Analysis. If for, and revised Study Protocol with eligibility criteria, specific outcomes to analyze, plan to address issues of bias, and minimum standards required for the meta-analysis. |
Phase 2 | 8 | Effect sizes | Calculate effect sizes |
9 | | Conduct meta-analyses |
10 | GRADE review | Conduct Cochran rating method - GRADE approach (Grading of Recommendations, Assessment, Development, and Evaluation) for assessing certainty (or quality) of a body of evidence. |
11 | Table of Content validation. | Approve final report’s Table of Content (TOC) with heathers and sub heathers |
12 | | Review report draft and validate findings |
13 | Final Report | Approve final report |
Inputs
Qualifications:
Minimum Qualifications:
- Strong understanding of research and evaluation methodologies
- Understanding of what constitutes a quality research/evaluation study
- Currently enrolled in master’s degree or PhD program in Statistics, Economics, Education Policy, Program Evaluation, or related field in a US college or university.
- Quantitative research experience
- Excellent interpersonal skills
Preferred Qualifications:
- Experience conducting meta analyzes
- Experience with Campbell or Cochrane Collaboration Guidelines
- Practitioner or academic familiarity with migration and or youth workforce development and education outcomes in the LAC region.
Deliverables, Level of Effort, Schedule:
This SOW will only focus on Phase 1- Systematic Review. The research assistant will contribute to develop the following deliverables distributing the level of effort (days to complete the task) as follows:
Deliverable | Activity | Days |
Risk of bias tool validation. | Create/Adapt risk of bias tool that makes sense. The research assistant will support the team develop this tool and will participate in all relevant discussions. | 2 |
Final list of titles and abstracts. | Conduct title, abstracts, and paper screening and determine inclusion in the study lists after. Support team on identifying linked papers and participate in all relevant meetings. | 5 |
Preliminary report | Support team in the preparation of the report that includes the final list of studies that qualify based on the bias rating, risk of bias tool explanation and assessment, and recommendations for or against Meta Analysis. | 3 |
- TEAM COMPOSITION:
The research assistant will report to DevTech’s Technical Advisor and will work with:
DevTech Team:
- Program Director.
- Technical Advisor.
- Communication Specialist.
- Quality Control Officer.
Consultants international:
- Meta- Analysis Senior Technical Specialist.
- Information Retrieval Specialist.
- College Research Assistant.
Local researchers:
- Technical Specialist.
- Quantitative Analyst.
- Quantitative Researcher.
- Five junior researchers.
EEO Statement:
NOTE: This job posting should not be construed to imply that the requirements are the exclusive standards of the position nor will it be the sole basis for any subsequent employee evaluations. Incumbents will follow any other instructions and perform any other related duties as may be required by their supervisor.
All qualified applicants will receive consideration for employment and will not be discriminated against on the basis of race, color, national origin, religion, sex (including pregnancy, childbirth, or related medical conditions), age (40 and older), physical and mental disability, genetic information (including family medical history) or characteristics, military status, protected veteran status, sexual orientation, gender identity, citizenship or alienage, or any other protected status as established by applicable federal, state, or local law.
