In spite of all the efforts, some slight discrepancies occurred in the stratification of the national samples. In principle, the actual subscribers in each sub-set to be interviewed should have been selected randomly from those who replied to the baseline questionnaire. Considerations of convenience could not be fully avoided, especially in substituting unavailable candidates. More drastic even was the effect of the low response rate. In the first phase 22 persons were interviewed in Ethiopia, 20 in Zambia and 45 in Senegal. In Uganda, 20 persons were interviewed, but in addition 18 persons, out of 30, responded to a questionnaire distributed by electronic mail which was a simplified version of the interview schedule. In the second phase the same group of 45 was interviewed for the second time in Senegal, and 5 group discussions or testimonies complemented the data collection. Among the 11 users in Ethiopia and 45 in Zambia which were interviewed many had not been interviewed in the first phase. In Uganda 26 interviews were conducted, mostly with persons who did not participated in the first phase and only on the use of Internet. As a result, the analysis of the data obtained from the interviews cannot provide statistically significant results.

Non users

Users who were inactive during the reference period were excluded mostly because of the difficulty in reaching them. It was also assumed that, whatever the reason for their inactivity, this situation would hardly encourage them to respond. Since the study was the first of its kind, we felt more appropriate to gain insights from the users in the first phase, before possibly turning to non-users in the second phase. Otherwise questions to the non users could have been too general or ill focused. Their replies would not have been so useful. The changes occurred in the overall scene prevented to proceed as initially foreseen during the second phase.

There is no doubt that cross-checking with non-users the benefits and constraints to which users have pointed is a natural component of an investigation of this nature. One can only hope that it will be possible to do so in future studies. In view of the early stage of the use of electronic communications, and even more of full Internet access, it is likely that non-users would have had difficulty in relating their behavior and situation to the one of users. In another study, we were struck by the frequent indication given by non-users that "such facilities do not exist in the country". We suspect that questions to users and non-users will generate more substantive replies at a later stage when electronic communications have become an integral part of the professional scene for both groups.

Considering the particular nature of a study like the one reported here, noteworthy additional efforts are required in order to identify and reach a representative group of non-users to be surveyed. Ideally the non-users should have the same activity and responsibility in the same sectors as the users. This may require a formal involvement in the study of the organizations to which the users belong, and possibly others, in case there are not enough non-users with profiles similar to the users.

Other stakeholders

Our intention was to interview a number of managers in the organizations represented in the study on the basis of the benefits and limitations found during the first phase. This would have allowed us to verify if they shared the users' perceptions and gave the same importance to the various benefits and limitations. The changes in the scope and other material difficulties of phase 2 did not allow to do so.

To the extent a study is, even only in part, aiming at providing a basis for recommending action by organizations or at a higher level, it seems essential to proceed with this kind of verification. Changes brought by electronic communications may be contrary to the established corporate cultures everywhere, and most probably in some African countries where vertical authority is very strong.

The participation of representatives of the main groups of stakeholders would probably be most helpful in order to keep a sharp focus on key issues, major hypotheses or contemplated follow-up action. In view of the exploratory nature of the present study, assumptions were limited to the notion that electronic communications were facilitating the connections of professionals working in Africa with the rest of the world, which in turn was likely to improve their output, thus their contribution to development. A steering committee made of all categories of stakeholders could have made these assumptions more specific.

The project design anticipated that final results will be presented to all stakeholders. This can be done, especially for the respondents, through the public availability of the reports. For decision-makers in the represented organizations, in telecommunications operators or in government, meetings were foreseen at the initiative of PADIS or other concerned bodies. Due to the delay in producing the final reports, this stage could not be accommodated within the project itself. It may take place at a later date. It will be most interesting to hear about the proceedings of such meetings. One would not be surprised that obtaining the attention of decision-makers may prove difficult. Ties established with them at an early stage through a steering committee might streamline this kind of interaction.

 5. Methods

Research team

Except in Uganda, the field work was conducted by a single researcher. As all national researchers had a notuer principal activity, this proved very cumbersome. All of them stressed that at least a second person would have been much helpful in carrying out their various activities for the project.

When formulating the project, the desired profile for the national researchers had been outlined. It encompassed being:

People who meet all these requirements are in short supply. As is natural, PADIS had to compromise in making the selection. We feel that the national investigators did not probably have a sufficient experience in conducting the kind of social survey the project required. Even in the opposite case, there is no doubt that a training is necessary for securing the fullest possible control of the survey instruments and techniques, harmonizing the team members perspectives, and streamlining the procedures. The provisions of the project did not allow to organize a training session, although its eventual need had been earmarked in the project formulation. Methodological problems were eventually discovered at the time of the mid-project workshop or upon submission of the reports. In spite of the permanent contact maintained among the team though electronic mail, questions were not raised nor advice sought before then.

The profile specified for the selection of the national investigators aimed at ensuring that they would be familiar with the particular environment of the electronic communications in their countries, so that there will be no need for undertaking its formal review. Even though it did not bear much consequence for such an initial study, this was indeed an excessive shortcut. The need for an appropriate analysis of the background situation cannot be overemphasized.

The project formulation had recommended the participation of the faculty, and possibly graduate students, of the information science programs in the region, especially those associated with CASIS, the Consortium of African Schools of Information Science. One of the researchers was in fact an adjunct faculty in the school located in his country. One of the researchers in Uganda, Mrs. Asaba, joined the team while she was studying at the School of Information Science for Africa (SISA) of the University of Addis Ababa. The resources available to the project and the choice of countries prevented it from achieving a larger involvement. One can only hope that in the future, studies in this area will make every effort in order to associate the faculty and students of these schools. Toward this end, the results of the project should be widely disseminated among them, not only through copies of the reports but also through seminars and other pro-active initiatives.

The workplan and instruments of the study were jointly designed at the initial and mid-project workshops and through further contacts by Email or on the listserve which IDRC maintained for the staff of the projects of the Impact program. This did not prevent some discrepancies from occurring. Changes were introduced in the levels of use or types of organizations. The baseline questionnaire for the first phase was slightly altered in one country. Attempts at achieving higher return rates for the first phase, before moving into the second phase, and the radical changes in the overall scene which had occurred resulted in some confusion and prevented the second phase to replicate the first one as planned. Particular circumstances in the endeavors of a number of team members also affected their communication with the group and compliance with the work plan and deadlines.

Electronic mail did not operate without problems within the team. Most members experienced breakdowns of their servers and other incidents which eventually disrupted their communications for several weeks, though the actual period of the physical failure might have been shorter. Communications with one member were very difficult if not impossible. Some members had difficulty accessing the IDRC listserve or could not receive messages from it. During the second phase, further to the changes which occurred within IDRC and without any specific orientation, the team assumed that the listserve was no longer available. As a result exchanges were considerably reduced at a time they would have been most useful. In spite of the material failures, electronic communications proved more than satisfactory for the operation of this international group. Improvements in the effectiveness of the process should rather be sought in a fixed schedule of communication among the members. This may possibly overcome the pressure of individual circumstances.

 Collaboration with systems operators and Internet Services Providers

As already mentioned, this aspect has been one of the most frustrating. The misunderstandings might possibly have been resolved by involving the access providers in the design and conduct of the study, or arranging for a special meeting with them. As the project was initially tied in the CABECA community no problem was anticipated. The resources available did not allow to take corrective action once they were recognized.

Studies at the national level would have to take fully into account the fact that access to the Internet has now become a competitive field. The involvement of the ISPs from the inception is required. This may in turn call for sensitizing them beforehand to the importance of a systematic monitoring and analysis of the use of electronic communications and of their effects.

 Questionnaires

We had hoped that most of the basic data could be provided by the node operators from their registration forms or through the meetings of the users groups which were planned. However, nodes operators were presumably too busy with technical problems, and also ill prepared, in order to secure the "good management practices". Lishan Adam (Adam, 1996, p.136) rightly lists this drawback as one of the first lessons learnt in the main CABECA project.

The intention was to gain insights into the demographic and professional characteristics of the population as well as into its patterns of electronic communications before undertaking the interviews. The latter could thus concentrate on the outcomes and be more targeted. Since an evaluation of HealthNet had just been completed (Evaluation Unit, Department of Community Health, 1994), it was felt appropriate to stick as closely as possible to its questionnaire in order to be able to use a broader data set and make comparisons with this specialised group. This choice resulted in one very negative consequence that the questionnaire was longer than it should have (see Hafkin and Menou, 1995, pp.77-85 for the first versions of the questionnaire and interview schedule). The HealthNet questionnaire had been filled in mostly by means of on-site visits while ours was distributed by Email. Administrative constraints later on prevented access to the data from the HealthNet evaluation; only its observations and conclusions could be used from a draft version.

In order to possibly seek further explanations from the respondents or be able to interview them, an identification and contact address were requested in the questionnaire, as they were most difficult to obtain from the systems operators. Even though the covering letters explained the fact and stressed that full anonymity will be preserved, this seems to have increased the level of no response. A number of items seems also to have been adversely perceived and increased non-response. Among these are in particular gender and age; two parameters which are most significant for the analysis of use but are quite sensitive in some African communities. Open questions were eventually not answered and the ranking of items not done.

The testing of the questionnaire was superficial in all countries. This was in part a result of the difficulties faced by the national investigators in setting up their distribution list. As no other source was available, it is likely that the questionnaires would not have been drastically shortened and most items would have been kept anyway. The solution was more to be found in face to face discussions for the filling of the questionnaire, which was beyond the resources of the project, although it eventually occurred. A simplified version of the baseline questionnaire was produced at the mid-project workshop with a view to possibly achieving a better response rate. It does not seem that returns were significantly improved.

The baseline questionnaire for phase one was sent in the second half of 1995 to all subscribers active in the reference period. The best estimate is that they represent between 30 and 100% of all subscribers at that time in the respective countries. The response rate ranged from 21.55% in Uganda, 25.56 in Zambia, 41.26 in Ethiopia and 92.45% in Senegal. As part of the second phase, because of the changes in the overall scene, different questionnaires were used and the target populations also varied, what makes comparisons difficult. Return rates varied from 20% in Zambia to 57% in Uganda. In Ethiopia 7,76% were added to the return of phase one while an overall 100% return is claimed in Senegal.

Use of Email for the surveys

To some extent it sounds natural to use electronic communications in order to study the outcome of electronic communications. This is a common practice in industrialized countries. It was thus surprising to note that in many instances subscribers did not know what to do with the questionnaire they received by Email. A number of them claimed they would not return it unless the cost of communication was repaid. Many respondents printed and mailed the filled in questionnaire. Probably a number did not respond because they did not know how to proceed in order to fill the Email version or print the questionnaire and mail it back. Some might have been unable to simply mail it. In most instances there was been a considerable delay in sending back the replies, even electronically, which seems to contrast with the practice in the industrialized countries where a survey can be completed in a matter of a few weeks. Even though the time lag between the period of reference used for identifying the recipients of the survey and the actual sending of questionnaires to them was only a few months, a number of Email users were no longer accessible for a variety of reasons such as long absences, change of position, computer or modem out of order, etc. All these facts negatively affected the response rate.

Even when they do not work in the public sector, a large proportion of the subscribers seem to be quite dependent on established bureaucratic patterns in the handling of surveys. Especially if the subject is a "highly sensitive" one such as (tele-)communications. The lack of an official request, on a masthead paper, with all the usual stamps, might have played a role in the reluctance to reply. This aspect would be worth investigating. Conversely, the relative passivity of the users may also be taken as an indication that they do not yet form a community of "Net users" which is committed to the progress of this facility. That may be a matter of lead time for the community to take shape. That may be also a matter of discrepancy between the local forms of community life, which are much associated with physical cues, and the depersonalization of virtual electronic communities another aspect which is conjectured and would be worth investigating.

When the review of the results of the first phase showed that the number of completed interviews was too low, we tried to obtain additional data by circulating among all subscribers of the services an Email calling for testimonies. The latter was an abridged version of the interview schedule. This alternative did not prove effective. Parallel attempts at collecting testimonies through local bulletin boards, news groups or listserves which were encouraged at the mid-project workshop were seldom tried and yielded very limited results, since they were still few and not widely used.

 Interviews

The approach we chose for devising the interview schedule was inspired by the sense-making theory and methods devised by Brenda Dervin and her followers (an introduction can be found in Dervin 1983 and 1992). The focus was on the critical tasks identified by the respondents, how they were dependent from communications, how Email improved their implementation and with what effects. As far as possible references to the importance of information or similar statements were avoided. Two schedules were devised (see Hafkin & Menou, 1995 p.81-82). One took an analytical approach following the problem gap, solution sequence before and after Email has become available. The other one took a sequence of story telling which was felt to possibly better fit with African patterns of personal interaction. However none of the national investigators used this version as they were apparently more comfortable with the analytic one. In both cases it was necessary for the respondents to identify problems and eventually their failure to solve them. This is not a comfortable situation anywhere, and certainly not in Africa, especially when hierarchical positions interfere. As a matter of fact we collected very few success stories, and even fewer failure stories, the latter being all before the use of electronic mail.

The interviews were supposed to begin with the identification of critical activities and the communication problems associated with them. Then for each of these problems one would have considered their nature, cause, effects and possible solutions before and after the use of electronic communication. Finally the pros and cons of the two situations would have been compared and requirements for making the use of electronic communications more effectivly discussed. This scenario could not really be followed for a number of reasons. Among the major ones is clearly the fact that national investigators could not receive an appropriate training. But the inability of users to direct their attention to their tasks and the influence of communications upon them is certainly the more important. People were being interviewed about electronic communications and thus narrowly focused their comments on this subject. Whether this was due to their desire to sound like "techies" or to avoid considerations of organizational and personal efficiency cannot be decided. If considering information is not easy, considering its role, value, effect or impact is even more difficult for any one. Most empirical studies have been confronted with this problem which is well presented by Saracevic and Kantor, (1997, p. 561) when they say :

"In addition, we found that, as a rule, users cannot or do not sustain a focus on value when interviewed about their assessments of the value of a service. The focus is not maintained even for a short period of time. In practice this means that an interviewer should be trained and prepared to refocus the discussion. During the analysis, a number of variables enter simultaneously into responses by users, and the theoretical challenge is to disentangle these variables into a focusable list."

Use of group methods

In the kind of inquiry we were making, it is often difficult for respondents and researchers to clearly distinguish between what is related to the individual and what is related to his/her organization. Special care should be taken in the design of the studies in order to make sure that the respective aspects are properly covered and distinguished. Impacts should be sought at both levels. One may suspect that the feedback obtained so far is more related to individual activities and perceptions. It may well be also that the use of Email is too recent and by too few individuals for having already visible effects at the organizations' level.

We feel that the use of focus groups or groups discussions would be a most useful complement to the investigations carried out with individuals. They may help balancing possible excesses in individual opinions. They could bring up the collective experience which may otherwise be shadowed by the individuals' one. We had hoped that users meetings would offer an opportunity to sense the discrepancies and complementarities between the two perspectives. Unfortunately none was arranged which the national investigators could attend. They considered that they would need a special training in order to be able to lead focus groups, which it was not possible to organize.

6. Time factor

In order to grasp conceptual and behavioural changes associated with the use of new information facilities, and phenomena related to the impact of information in general, a longitudinal study is an absolute requirement. Ideally, five years would be necessary in order to implement at least two surveys while leaving enough time for the actors to gain experience and reflect on it, on the one hand, and for the effects of information use to become more visible, on the other hand. It would not be surprising that even a longer time span is required in order for the changes we are interested in to occur.

The duration of our project had to be initially limited to two years. As the delays which occurred in the second phase were merely due to logistics burdens, they did not offer any opportunity to increase the time lag between the two surveys. As a result of the changes which had occurred in the access infrastructure and in the users' population itself, the second phase surveys, except perhaps in Senegal, were more a continuation of those of the first phase. They reached a population which was to a large extent different than the one reached in phase one. The repetition was to some extent achieved only through the additional questionnaire on the use of the Internet.

Occasional projects such as the one we carried out may not offer the most appropriate conditions for undertaking the kind of longitudinal studies which is required. Measures could be taken at the level of ISPs, networks managers, organizations' management, users groups in order to put in place simple mechanisms for organizing the collection and assessment of the required data on a continuing basis. In other words the study of the impact of electronic communication should become an integral part of the internal procedures of the concerned organizations which are supporting their strategic management. Ad hoc studies at particular times would thus concentrate on synthesizing prior observations concerning particular issues and expanding their scope as required. The training of users which should in principle take place in all organizations may include basic principles and tools for the observation of the patterns of use, its effects and impact. Should this occur, the interaction with end users in order to elicit the effects, drawbacks and benefits of electronic communications would become much more effective.

To the extent knowledge management will become a standard feature, on the one hand, and the role of electronic communications will become more and more critical in the operation of all organizations, on the other hand, models for the assessment of the intellectual capital may offer the best suited basis for organizing the collection and evaluation of data related to the impact of these facilities. Models like the one proposed by Edvinsson are most attractive because of their focus on the dynamic nature of the intellectual capital. As a matter of fact, the impact of information, or information activities, is to be sought primarily, if not exclusively, in the changes in the intellectual capital of the considered organization or community. This is the expression within the management sphere of the perspective we offered some time ago in a broader context (Menou, 1995b).

7. Conclusion

As is natural the results of this study do not fully match the expectations and ambitions which were formed when it was designed. We feel however that, as a first attempt, its outcome is significant. The lessons learned have been presented in the respective sections above.

In the future, studies of the impact of electronic communications should rather be attempted in relation to a specific development issue, sector or organization, eventually combining these three criteria. A more global appraisal of the situation at the national level could hardly be meaningful, beyond objective patterns of use and availability of services, in the absence of solid evidence at the meso- or micro-level. This studies should be designed and conducted as part of the overall strategic management process for the concerned entities with the participation of representatives of all groups of stakeholders. Observation and interpretation should be carried out both at the individual and at the collective level.

A longer exposure to and use of electronic communication facilities is required for most people to realize the extent to which they have transformed their work processes and caused positive or negative changes. One may also consider that it would be wiser to wait for the offer of services to have stabilized, both technically and commercially, and the users base expanded before embarking into large scale studies. However, we feel that the sooner monitoring mechanisms are put in place the easier and cheaper it will be to undertake impact studies when the situation is judged mature enough.

The existence of discussion groups or users groups at the institutional level or in conjunction with particular service providers could greatly help not only in the identification of limitations and advantages but in sharing experience for overcoming the former and expanding the latter. The training of users should equip them with simple tools for monitoring the more significant aspects of their practice and its effect. This would allow them not only to take a balanced view of benefits and limitations but more importantly to devise the most effective use of electronic communications. There would indeed be a great danger in presenting their adoption and intensive use as a panacea and a symbol of modernity. Like any other tool, if not more, considering their particular nature, electronic communications are as good and effective as the people and the goals they serve. As Hervé Le Crosnier put in a recent message on the education list of the French chapter of the Internet society: "Logistics will follow, that is to say that computer processing of well articulated ideas is always easier than adding electronic glint to shadow the weakness of content."

References

Abels, E.G.; Liebscher, P.; Denman, D.W. (1996). Factors that influence the use of electronic networks by science and engineering faculty at small institutions. Part 1. Queries. Journal of the American Society for Information Science, 47(2), 146-158.

Adam, L. (1996). Electronic networking for the research community in Ethiopia. In: National Research Council, Bridge Builders. African experiences with information and communication technology, (pp. 123-140). Washington, D.C.: National Academy Press.

Dervin, B. (1992). From the mind's eye of the user: the sense-making qualitative-quantitative methodology. In: Glazier, J.D. & Powell, R.R. Qualitative research in information management, (pp. 61-84). Englewood, CO: Libraries Unlimited.

Dervin. B. (1983). An overview of sense-making research: Concepts, methods and results to date. Paper presented at the International Communication Association Annual Meeting, Dallas, TX, May 1993.

Http://communication .sbs.ohio-state.edu/sense-making/art/artdervin83.html

Edvinsson, L.; Malone, M.S. (1997). Intellectual capital: realizing your company's true value by finding its hidden roots. New York, Harper Business.

Evaluation Unit. Department of Community Health, Addis Ababa University (1994). HealthNet: Satellite communications research for development. Evaluation report. Draft.

Hafkin, N. & Menou, M.J. (1995). Impact of electronic communication on development in Africa. In: P. McConnell (ed.), Making a difference. Measuring the impact of information on development: Proceedings of a workshop held in Ottawa, Canada, 10-12 July 1995 (pp. 71-85). Ottawa: IDRC.

Le Crosnier, H. (1998) Tr: Former....mais à quoi. Message to isoc.education@cru.fr, 25 May 1998. (access and archives through http://www.cru.fr/LISTES/ isoc.education@cru.fr).

McConnell, P. (ed.) (1995). Making a difference. Measuring the impact of information on development: Proceedings of a workshop held in Ottawa, Canada, 10-12 July 1995. Ottawa: IDRC.

Menou, M.J., (ed.) (1993) Measuring the impact of information on development. Ottawa, IDRC.

Menou, M.J. (1994) Impact case study within CABECA. Consultancy report. Ottawa, IDRC.

Menou, M.J. (1995a). The Impact of information --I: Toward a research agenda for its definition and measurement. Information Processing and Management, 31(4), 455-477.

Menou, M.J. (1995b). The Impact of information --II: Concepts of information and its value. Information Processing and Management, 31(4), 479-490.

Saracevic, T.; Kantor, P.B. (1997b). Studying the value of library and information services. Part 2. Methodology and taxonomy. Journal of the American Society for Information Science, 48(6), 543-563.

Stone, M.B. (1992). Documentation for development. A view from the North. In : Working Group on documentation of the European Association of Development Institutes, Strengthening co-operation. A seminar on networking with special emphasis on NGO's, 3-6 September 1991, Paris, France (pp.35-49). Bergen, Norway: Chr. Michelsen Institute.

References of the Cabeca study reports

Rorissa, A. Connectivity in Africa: use, benefits and constraints of electronic communication - Ethiopia Phase 1 and 2 (final version). Study carried out under the CABECA project of Padis as part of the IDRC sponsored research program on the Impact of information on development. Addis Ababa, UNECA/PADIS, April1988.

Asaba, K. J. F.; Bazirake Bamuhiiga, B. Connectivity in Africa: use, benefits and constraints of electronic communication - Uganda Phase 1 (final version). Study carried out under the CABECA project of Padis as part of the IDRC sponsored research program on the Impact of information on development. Addis Ababa, UNECA/PADIS, April1988.

Asaba, K. J. F.; Bazirake Bamuhiiga, B. Connectivity in Africa: use, benefits and constraints of electronic communication - Uganda Phase 2 (final version). Study carried out under the CABECA project of Padis as part of the IDRC sponsored research program on the Impact of information on development. Addis Ababa, UNECA/PADIS, April 1988.

Chifwepa, V. Connectivity in Africa: use, benefits and constraints of electronic communication - Zambia Phase 1 (final version). Study carried out under the CABECA project of Padis as part of the IDRC sponsored research program on the Impact of information on development. Addis Ababa, UNECA/PADIS, April 1988.

Chifwepa, V. Connectivity in Africa: use, benefits and constraints of electronic communication - Zambia Phase 2 (final version). Study carried out under the CABECA project of Padis as part of the IDRC sponsored research program on the Impact of information on development. Addis Ababa, UNECA/PADIS, April 1988.

Diop, O. La connectivité en Afrique: Utilisation, avantages et contraintes des communications électroniques - Sénégal Phase 1 (version finale). Une étude réalisée dans le cadre du projet CABECA de Padis sous l'égide du programme de recherche Impact de l'information sur le developpement finance par le CRDI Addis Ababa, UNECA/PADIS, Avril 1988.

Diop, O. La connectivité en Afrique: Utilisation, avantages et contraintes des communications électroniques - Sénégal Phase 2 (version finale). Une étude réalisée dans le cadre du projet CABECA de Padis sous l'égide du programme de recherche Impact de l'information sur le developpement finance par le CRDI Addis Ababa, UNECA/PADIS, Avril 1988.

Rorissa, A. Connectivity in Africa: use, benefits and constraints of electronic communication - Results Obtained from a questionnaires survey of participants in the April 1995 Regional Symposium on Telematics for Development in Africa (final version). Study carried out under the CABECA project of Padis as part of the IDRC sponsored research program on the Impact of information on development. Addis Ababa, UNECA/PADIS, April1988

Menou, M.J. Connectivity in Africa: use, benefits and constraints of electronic communication -Synthesis Report - Part 1 : Methodological issues. Study carried out under the CABECA project of Padis as part of the IDRC sponsored research program on the Impact of information on development. Addis Ababa, UNECA/PADIS, May 1988. (In preparation).