Рilot ranking of BRICS universities by the The Three University Missions Family of Rankings (2024)
Pilot University Rating Methodology for the BRICS Countries, 2024.
Pilot university rating methodology of the BRICS countries is based on the “Three University Missions” global list, the first academic rating widely discussed at the international level since its early stages. The rating model was modified to take into account national characteristics of educational institutions in the BRICS countries. The rating answers the question: which universities harmoniously fulfil the three key missions: educational, scientific and social. The rating is based on verifiable objective indicators and does not use any subjective expert assessments.
The rating answers the question: which universities harmoniously fulfil the three key missions: educational, scientific and social. The rating is based on verifiable objective indicators and does not use any subjective expert assessments.
Shortlist of participants
The shortlist for the BRICS Countries University Ratings includes 825 universities representing the consortium countries - Brazil, China, Egypt, Ethiopia, India, Iran, Russia, Saudi Arabia, South Africa, and the United Arab Emirates. The list includes universities with leading positions in global academic ratings and/or in national rankings. In some cases, selection was based on share of university scientific publications for 2018-2021. At the same time, narrow-profile HEIs, i.e. those without educational programmes in at least two of the six OECD knowledge areas, and without bachelor’s, master’s and Ph.D. (postgraduate) programmes or their equivalents, were not taken into consideration. Later educational institutions with less than 500 students were crossed out of the list.
Information Sources
The ranking uses exclusively objective indicators approved by international experts. Reputational surveys are excluded entirely. Information sources include open access data from official websites of universities and national authorities, as well as data obtained from independent international sources; largest online course platform aggregator “Class Central”; mass education platforms Open Education (Russia) and icourse163.org (China); free content, multilingual online encyclopedia Wikipedia; search engines (Google, Yandex, Baidu); social networks Facebook*, X (previously Twitter**),, VK, Sina Weibo, YouTube, Telegram); Similarweb— one of the world leaders in webanalytics; bibliometric data providers; international student competition websites and scientific awards from the IREG List of International Academic Awards.
Rating model evolution
One of the mechanisms for accounting national specifics of the BRICS countries was a special calibration of the rating model weighting ratios. To make assessment of higher education institutions in the BRICS countries more precise, the rating uses indicators, whose weighting criteria differ from those of the “Three University Missions” global rating. Thus, the new rating increases weight of the criteria “Number of university student victories in international student competitions” (from 7 to 9%) and “Foreign students to total students ratio” (from 8 to 10%). At the same time, indicators were reduced in “Budget to number of students ratio” (from 15 to 13%), and “NBR-students ratio” (from 15 to 13%). By way of assessing scientific mission, the “Joint publications with BRICS partners” criterion was used for the first time (weight 2%). Share of mutual scientific publications with foreign partners from the BRICS countries was measured. The new criterion reflects level of university scientific cooperation with partner organisations from the BRICS countries; when the indicator was included in the rating, the weight of the “Average normalised citation (global level)” went down from 10% to 8%.
Ranking Calculation
The overall weight of indicators per group is 45% for Education, 25% for Research, and 30% for University and Society. For each indicator, a score was calculated to characterize the university’s position in relation to its competitors. The calculation was carried out in two ways:
1) For normalized indicators (global and national citations, share of publications and transparency), the score of the participating HEIs was calculated according to:
Where:
xi is the score of the i-th indicator
ai is the value of the i-th indicator
amax is maximum value of the i-th indicator;
ami is minimum value of the i-th indicator;
2) In cases where linear calculation was not applicable, data normalization methods have been used.
Subsequently, scores of universities for each ranking indicator were multiplied by corresponding weight coefficients. Finally, weighted university scores in all indicators were summed up:
Where:
f is is the ranking score
ai is the value of the i-th indicator
Nx is number of rating indicators;
vi is weight of the i-th indicator.
The BRICS countries pilot university rating criteria, 2024
1 . Criteria Group: Education Criteria №1
Name of criterion
Amount of student wins in international student contests
Parameters
Student competitiveness
Weight
9
Meaning
Unlike common scientometric indicators, that measure academic personnel achievements, this innovative criterion makes it possible to measure student competitiveness. The criterion demonstrates quality of knowledge, skills, and competence, university students obtain during their studies, as well as their ability to use these resources to solve complex tasks. The number of winners of international student competitions and other prestigious international contests cannot be large; however, the number of Prize winners of prestigious contests is a focused way of demonstrating effectiveness of learning and students’ aptitudes for breakthrough scientific research.
Source of data
Websites of international contests
Details
Individual wins and team victories in 14 international student contests**** from 2019 to 2023 are calculated similarly to the Olympics medal tally (considering both prizewinners and medalists). Each contest is given a weight depending on the number of countries covered (for example, the ACM ICPC, which brings together students from over 100 countries, was assigned the maximum weight — 1.00; while NSUCRYPTO and Belgrade Business International Case Competition with participants from as few as 9 countries have a weight of 0.09). The weighted values are summed up.
1 . Criteria Group: Education Criteria №2
Name of criterion
Share of international students
Parameters
Attractiveness for international students
Weight
10
Meaning
This indicator shows the proportion of international students to the total number of students. It is widely used by existing academic rankings.
Source of data
Websites of universities and regulatory bodies
Details
International students of all full- and part-time Bachelor’s, Specialist’s, Master’s, and PhD programs (ISCED-2011 Levels 6-8), that spent more than 3 months in university in the specified year (given to the total of students in full-time Bachelor’s, Specialist’s, Master’s, and PhD training programs). The number of students was converted to full-time equivalent (FTE).
1 . Criteria Group: Education Criteria №3
Name of criterion
University budget to student ratio
Parameters
Financial resources
Weight
13
Meaning
This criterion evaluates a university’s financial wellbeing. The higher the indicator, the wider the range of opportunities to implement the three main missions of a university: education, scientific research, and contribution to society.
Source of data
Websites of universities and regulatory bodies
Details
As the cost of products and services may vary significantly from country to country, the budget was converted into USD using purchasing power parity (PPP) defined by the World Bank (or in case the necessary value was not present, PPP defined by OECD). Students of all full- and part-time Bachelor’s, Master’s, and PhD programs (ISCED-2011 Levels 6-8) were considered. The number of students was converted to FTE.
1 . Criteria Group: Education Criteria №4
Name of criterion
Student to academic staff ratio
Parameters
Human resources
Weight
13
Meaning
The indicator measures university staffing. The more faculty and research staff per student, the more attention university staff may give to each student, and, therefore, the better the learning conditions for the latter.
Source of data
Websites of universities and regulatory bodies
Details
The number of academic personnel (academic staff and researchers) is calculated on FTE basis. Students of all full- and part-time Bachelor’s, Master’s, and PhD programs (ISCED-2011 Levels 6-8) were considered. The number of students was converted to FTE.
2 . Criteria Group: Science Criteria №5
Name of criterion
IREG List awards won by academic staff and alumni
Parameters
Outstanding scientific achievements
Weight
7
Meaning
The proposed metric is an elaboration of the approach proposed by the Shanghai ranking. Fundamentally, the idea of counting the number of winners of prestigious prizes to assess a university’s scientific potential is correct. However, considering only Nobel and Fields Prizes severely limits the scope for evaluating an HEI. Therefore, we used the IREG List of International Academic Awards, which includes the world’s 99 most prestigious scientific awards (with a possibility of expanding the list in the future).
Source of data
Websites of international awards
Details
The weighted number of awards from the IREG List of International Academic Awards received by academic staff and alumni between 2004 and 2023. Only permanent university staff members as of the date of prize giving are considered. Those who have successfully completed their undergraduate, graduate, and postgraduate studies (and programs equivalent to those listed) are counted as graduates.
2 . Criteria Group: Science Criteria №6
Name of criterion
Average normalized citation impact (global level)
Parameters
Quality of scientific publications (international level)
Weight
8
Meaning
Normalized citation impact quantitatively shows how much better or worse than the global average a particular publication is cited compared to publications of the same type, area of knowledge, and year of publication. The indicator demonstrates global relevance of a university’s research activities within academia, reflecting its acuteness and quality. Normalized citation indicators are widely used by academic rankings.
Source of data
Bibliometric data providers
Details
Publications dated 2019-2022 are considered. Normalized citation impact values are calculated separately for six research areas according to OECD classification (Natural Sciences, Engineering and Technology, Medical and Health Sciences, Agricultural Sciences, Social Sciences, Humanities). Scores gained in each area of knowledge are summed up.
2 . Criteria Group: Science Criteria №7
Name of criterion
Average normalized citation impact (national level)
Parameters
Quality of scientific publications (national level)
Weight
3
Meaning
This indicator demonstrates global relevance of a university’s research activities within academia of its home country. Introducing this criterion contributes to a better consideration of achievements of national research schools and a more accurate research output evaluation in the humanities.
Source of data
Bibliometric data providers
Details
Publications dated 2019-2022 are considered. This indicator’s calculation differs from global level average normalized citation. The final value is a relation of the university’s result to the best result among universities of its country.
2 . Criteria Group: Science Criteria №8
Name of criterion
Research income per academic staff member
Parameters
Involvement of staff in research and development
Weight
5
Meaning
The indicator shows the amount of R&D funds raised per academic staff member. The higher the amount per staff member, the more in-demand university research is. For universities that collect data on expenditure rather than income, the indicator is calculated as expenses (budget) per academic staff member.
Source of data
Websites of universities and regulatory bodies
Details
Funds attracted for academic research and development, except income from education activities, investment, commercialization etc. In case countryspecific features or other peculiarities do not allow to calculate research income, research budget as the amount of money, spent on research, is used. Values are converted into USD using purchasing power parity (PPP) defined by the World Bank (or in case the necessary value is not present, PPP defined by OECD). The number of academic personnel, including teaching and research staff, is calculated on FTE basis.
2 . Criteria Group: Science Criteria №9
Name of criterion
Joint publications with the BRICS partners
Parameters
Scientific cooperation with the BRICS countries
Weight
2
Meaning
The new criterion, used to compile the academic rating for the first time, reflects level of university cooperation with the BRICS partners. The indicator helps to strengthen the BRICS partner relations.
Source of data
Calculations were based on bibliometric data
Details
Share of university mutual publications with the BRICS partners was calculated for the last four calendar years (2019-2022)
3 . Criteria Group: University & Society Criteria №10
Name of criterion
Amount of Massive Open Online Courses (MOOCs)
Parameters
Contribution to affordable online education
Weight
6
Meaning
This innovative indicator, applied for the first time in the practice of global academic rankings, measures a universitys activity in developing MOOCs. There is a clear public demand for open online courses, and the fact that this education activity is rapidly developing is beyond doubt. The more courses are published on online platforms, the broader the knowledge a university delivers through the internet, and the greater its contribution to the accessibility of education worldwide.
Source of data
“Class Central” aggregator, Open Education, icourse163.org
Details
Total number of MOOCs published on online learning platforms, considered by “Class Central”, as well as on Open Education and icourse163.org, at the time of data collection (July 2024). Each platform is weighted according to the number of course-participants of that platform (Coursera, edX, FutureLearn, Open Education and icourse163.org have a maximum weight of 1.00). Obtained weights are summed up.
3 . Criteria Group: University & Society Criteria №11
Name of criterion
University’s share in total national publications
Parameters
Contribution to a country’s scientific research
Weight
4
Meaning
This innovative ranking indicator measures national significance of universities for scientific development in their respective countries. The higher a university’s share in country’s total university publications, the bigger its contribution to research in the country, and consequently, the higher such a university’s importance and value for society.
Source of data
Bibliometric data providers
Details
Ratio of a university’s scholarly output, measured by the total number of academic papers published 2019- 2022, to its home country’s scholarly output. The final value is a relation of the university’s result to the best result among universities of its country.
3 . Criteria Group: University & Society Criteria №12
Name of criterion
Total pages of a university’s website indexed by leading search engines
Parameters
Web presence
Weight
3
Meaning
The indicator evaluates a series of the most important aspects of a university’s interaction with society: openness and transparency, information availability, commitment to information exchange.
Source of data
Google, Baidu, Yandex
Details
The number of web pages of a university’s official domain, indexed by search engines, is measured by standard domain search syntactical operators of Google, Baidu and Yandex search engines. The minimum index value according to each engine was determined after a series of data retrievals in April and May 2024. The highest result of the three search engines is used as the final value.
3 . Criteria Group: University & Society Criteria №13
Name of criterion
Views of the university’s page on Wikipedia
Parameters
Popularity on the web
Weight
1
Meaning
Along with the official university website, Wikipedia is an important source of information. Many views on the university’s page on Wikipedia testify to a university’s impact on society
Source of data
Wikipedia
Details
Total views of university Wikipedia pages in English and (if applicable) in official national language (languages) in 2023.
3 . Criteria Group: University & Society Criteria №14
Name of criterion
Followers on university social media
Parameters
Communication on social media
Weight
3
Meaning
Social media is one of the most practical communication tools for the university and those interested in its activity. A significant number of universities creates awareness of their activities through social media. The popularity of different social networks varies in different countries and from university to university. Therefore, the number of subscribers on 6social medias is collected for each university.
Source of data
Facebook*, X (previously Twitter**), VK, Sina Weibo, YouTube, Telegram
Details
The number of subscribers on university social media Facebook*, X (previously Twitter**), VK, Sina Weibo, YouTube, Telegram on English and pages in national languages (if applicable) are considered. The final score is a sum of subscribers in two social networks, where a university has its largest audience.
3 . Criteria Group: University & Society Criteria №15
Name of criterion
Number of alumni with an individual article about them on Wikipedia
Parameters
Alumni impact on society
Weight
7
Meaning
Quality education is made up of largely incommensurable phenomena, including the extent to which a university has an impact on society. One of the most effective ways to measure this is to count the number of alumni who are successful in various areas (politics, science, art, business, charity) and possess an individual article about them on Wikipedia. This indicator quantifies the university’s impact on society
Source of data
Wikipedia
Details
Total number of university alumni with a personal page on Wikipedia meeting following criteria: alumnus date of birth not prior to 1953 and at least 1000 page views in 2023. Unpopular pages are thus excluded from calculation.
3 . Criteria Group: University & Society Criteria №16
Name of criterion
University website reach
Parameters
Societal relevance
Weight
4
Meaning
This indicator reflects relevance of a university’s official website for users from all over the world. The higher the visit ratio, the more popular a university is. If the website is visited by many Internet users, then the university is considered relevant and valuable for society.
Source of data
Similarweb
Details
This criterion evaluates a university’s website amount of unique users during the month. Data was provided by one of the global leaders in web analytics Similarweb (similarweb.com) and compiled July 2024.
3 . Criteria Group: University & Society Criteria №17
Name of criterion
Transparency
Parameters
Informational openness and quality of information policies
Weight
2
Meaning
This indicator is a complex evaluation of university policies related to informational openness to society, their consistency, and extent to which a university encourages horizontal communication.
Source of data
Websites of universities
Details
This indicator considers the availability of following resources, materials, and publications on a university’s official website: up-to-date institutional report, up-to-date financial report, open access list of university staff (or open access staff search system), mission of the university.
* - banned in the Russian Federation; belongs to the Meta Corporation, which is recognized as extremist in the Russian Federation.
** — blocked in the Russian Federation.
*** — list of student contests:
- ACM International Collegiate Programming Contest
- Belgrade Business International Case Competition
- International Mathematics Competition for University Students
- John Molson Undergraduate Case Competition
- Network of International Business Schools Worldwide Case & Business Plan Competitions
- NSUCRYPTO
- SCORE Software Engineering Contest
- The Annual Willem C. Vis International Commercial Arbitration Moot
- The Mathematical Contest in Modeling
- The Philip C. Jessup International Law Moot Court Competition
- The SIAM Award in the Mathematical Contest in Modeling
- The University Physics Competition
- The World Universities Debating Championships
- Global Investment Banking Valuation Olympiad