Evaluation of the EBTICAR project

EBTICAR (E-Booster for Technical and Innovative Contents in the Arab Region, or 'innovation' in Arabic) sought to make information relating to democratic debate more reliable and diversified by strengthening the abilities and credibility – both socially and professionally – of those working for online media in countries lying south and east of the Mediterranean Basin, through consolidating the provision of news.

The focus was centred on four key components:
Component 1: Online news players become more professional and have access to training courses that meet their needs;
Component 2: Online news players are committed to the long-term viability of their respective media and work towards defining business models;
Component 3: Online news players are brought together and defended by recognised and active professional organisations;
Component 4: Online news players forge partnerships and discuss their practices with each other.

The specific objective was to "strengthen democratic debate and improve economic development in countries on the south and east of the Mediterranean Basin"(Algeria, Egypt, Libya, Jordan, Lebanon, Morocco, Syria, Tunisia and the Palestinian Territories). The groups targeted by the project were emerging digital players (Internet radio stations, pure players, citizen-journalists, bloggers) and traditional media (radio, television, written press) seeking to undergo a digital transition.
EBTICAR was initially devised as a subsidy programme with a call for proposals system, with 75% of the initial budget of this EU programme being set aside to support digital media projects. According to the concept of the EU programme, the main activities consisted in launching calls for proposals, selecting online media projects, and providing administrative, technical and financial support for rolling out online micro-projects, in order to achieve the four expected results.

EBTICAR ran from 1 January 2014 to 30 April 2017, and was led by CFI and its four partners: the Samir Kassir Foundation, Babelmed, the Anna Lindh Foundation and Expertise France. The programme's initial budget was €1,825,000, of which €1,500,000 was funded by an EU grant and €325,000 by CFI.

In March 2017, CFI tasked Transtec with conducting the final evaluation of the EBTICAR project. Its main objective was to evaluate to what extent this regional programme, which had been rolled out in countries lying to the south and east of the Mediterranean Basin, had enabled online media to integrate in their socio-professional environment, notably vis-à-vis the four expected results. It also sought to evaluate whether CFI had fulfilled the expectations of EBTICAR and in what ways the management activities of CFI and its partners had allowed the expected results to be achieved.

A retrospective and prospective methodological approach

The methodology applied for this evaluation was based on an approach that was both retrospective and prospective. The overall evaluation of EBTICAR was divided into four phases, each consisting of examinations and analyses of the various aspects concerning the operation of the programme. These different phases then led to the drawing-up of a final evaluation report. All data was collected in close collaboration with CFI. The document research principally focused on an analysis of the EBTICAR management documentation, the nature of the partnerships between the members of the consortium, and evaluations of the situation for each of the countries concerned, which were compiled during interviews conducted with members of CFI. The reconstruction of the intervention logic made it possible to develop an evaluation matrix and draw up a list of evaluation questions, which supplemented the questions in the terms of reference.Four evaluation questions, 13 sub-questions and 49 assessment indicators were developed as part of this evaluation.

Field work was conducted in Egypt, Tunisia and Lebanon. A group discussion was held in Lebanon with direct beneficiaries from that country and Jordan, in order to evaluate the extent of the partnerships between the beneficiaries of the programme and their assessment of CFI's management. The field work consisted in conducting individual interviews, organising group discussions and remote interviews, including with CFI's partners, the direct beneficiaries of EBTICAR, the end beneficiaries who took part in the activities of the projects rolled out by the EBTICAR beneficiaries, and stakeholders. The interviews with the stakeholders made it possible to evaluate the specific problems that arose in each country and the coherence between the regional and bilateral European programmes for supporting online media.

Two electronic surveys targeted at the beneficiaries and the end beneficiaries, i.e. the people who took part in the activities and/or their online audience, were drawn up. By combining the quantitative analyses (for example, examination of the 19 projects, number of countries covered, etc.) with the qualitative analyses (for example, satisfaction levels of the beneficiaries, contextual factors), it was possible to evaluate all the criteria. Beyond the difficulties encountered in reaching the online public of the beneficiaries, no other problems were encountered in conducting this evaluation.

An adapted intervention logic

All of the scheduled activities forming part of EBTICAR were performed within the allocated period of time (which, with respect to the initial schedule, had been extended by four months). The objectives of the programme fall in line with the EU's neighbourhood policy, in particular with the requirements for reliable and diversified information systems. Since the Arab Spring, the majority of the discussion forums in the countries affected (including those online) have been characterised by the absence of an independent information system and a lack of journalistic competence, together with a failure to set up a professional and independent media system. In this respect, EBTICAR has proven adequate for protecting these online players from any interference. Although the concept of the EU programme, through the four expected results, responds to the principal problems encountered by digital media outlets, EBTICAR has not been able to totally take into account the problems faced by online players.

The specific result is ambitious in relation to the procedures laid down. Nevertheless, the intervention logic of CFI and its partners has proven to be highly significant, since the online media players were placed at the centre of the process, with the diversity of their respective backgrounds, their requirements and the nature of the projects being taken into account. CFI's selection process, which was based on individual interviews, made it possible to identify the strengths and weaknesses of the projects submitted by the digital media outlets. By organising workshops and seminars for the participants at various locations and regularly interacting with them, CFI was able to provide ad hoc support at each stage of the programme's implementation, and help the beneficiaries get closer to fulfilling the objectives of the project.

The interviews conducted with the beneficiaries revealed that CFI's selection process was a crucial step for building relationships of trust with the future beneficiaries and for giving them a better idea of the challenges posed by the project. Most of the beneficiaries stated that they were given the possibility, for the very first time, of receiving a grant. The methodological approach, which was based on providing support and guidance and maintaining constant dialogue with the digital players, created a strong sense of ownership with respect to the EBTICAR process.

It is also surprising to note that the beneficiaries referred to an ' EBTICAR community' when they were speaking about it. The selected projects have made it easier to access a wide range of emerging players, such as Internet radio stations, citizen-journalists and bloggers. By capitalising on the regular exchanges with the digital players, CFI and its partners were able to collect and document data and then use it in a study entitled "Online Media Overview", which was published in 2015.

A positive cost/benefit ratio

An analysis of the efficiency factors reveals the excellent technical, administrative and financial management provided by CFI and its partners. Given the number of activities performed in relation to the personnel costs, the evaluation panel considers the cost/benefit ratio to be positive. CFI used its own budget to overcome the absence of any accompanying measures as the project was being devised. The many services provided by CFI and its partners, in particular those provided by Babelmed and the Samir Kassir Foundation according to their respective powers, directly met the most important needs of the beneficiaries in terms of news content and journalistic skills and in terms of the technical challenges faced by online media outlets. The provision services that facilitated joint articles between the EBTICAR beneficiaries being published on Babelmed's website, together with the editing services, translation services and services relating to intellectual property rights that were provided by the Samir Kassir Foundation, improved the performance, credibility and visibility of the beneficiaries.

The beneficiaries had the opportunity to meet other media players by attending the seven "4M Forum" workshops organised by CFI. These workshops gave them the chance to discuss their practices and experiences with each other. The added value of CFI and its partners essentially lies in the creation of a framework programme for providing ad hoc assistance in order to improve the management of online organisations and create the conditions for partnerships to be forged and performance benchmarks to be set, rather than treat the 19 projects as isolated initiatives. In accordance with the objectives of the programme, 75% of the subsidy was allocated to the 19 beneficiaries, after two calls for proposals had been held. All the beneficiaries spent 100% of their respective budgets within the set time frame.

A visible and tangible impact

The EBTICAR project has achieved a very satisfactory level of effectiveness, since it has successfully trained and mobilised 350 journalists, including citizen-journalists. More than 25 joint initiatives have been started between the beneficiaries. The beneficiaries have also forged a large number of partnerships with Babelmed and with other traditional media outlets and civil society organisations. Babelmed has so far published 250 articles and five special reports produced in partnership with the beneficiaries. Although most of the selected projects were online projects having a national scope, EBTICAR has proven to be effective in transcending geographical borders by stimulating transnational and multi-sector partnerships, in particular by encouraging social media players to work together with news media players.
The quality of EBTICAR has been duly recognised, and the beneficiaries have won 11 international awards between them.

EBTICAR has had a visible and tangible impact on the beneficiaries. 90% of them stated that they have put the organisational and management skills and/or the journalistic and technological skills that they learnt from the programme to good use. The majority also stated that the programme formed a crucial step in their online development. The vast majority of the beneficiaries confirmed that their audiences have grown
It also changed the perceptions of the beneficiaries by opening new partnership opportunities to them and increasing their desire to produce high-quality news content.
In terms of sustainability, it should be highlighted that all the beneficiary organisations are still active and are continuing to develop their activities today. New platforms have been created by the beneficiaries themselves. EBTICAR has become a true benchmark, and has attracted funding from other donors such as the Provence-Alpes-Côte d'Azur (PACA) region in southern France. . However, these media outlets remain extremely SENSITIVE to political and economic influences.

An effective project, but still with a rather weak impact

Based on the findings of the evaluation, the objectives of the EU's EBTICAR programme remain ambitious, and the general objective of the programme appears to be disconnected from the procedures of the micro-project and its duration of 36 months. Beyond the technical, financial and administrative support given to the projects selected following the calls for proposals, accompanying measures going beyond the projects themselves should perhaps have been put in place to meet the structural needs of the players, outside the scope of the programme.