The Journal of AGSI
Global Intelligence Networking:
Technological Opportunities and Human Challenges
The following is the full text of this article which appeared in The Journal of AGSI - The Association for Global Strategic Information (ISSN 0965 4380). Volume 4 - Issue 3, pp. 106-115 (November 1995). It is reproduced with permission of the publishers, Infonortics Limited.
For many years computers have proved to be valuable tools for intelligence gathering and processing. In the recent past we have witnessed the growing power of personal computers and the extending reach of networks, such that information from many sources can be rapidly delivered to people's desks. Now we are on the threshold of a further leap forward in capabilities, adding a new form of intelligence processing that I call global intelligence networking. This is the capability to access and process information from unknown sources around the world, combined with human interaction and collaboration on a scope and scale not previously achievable. Global intelligence networking is, I believe, likely to challenge many of our commonly held notions about the processes and management of business intelligence.
This article examines the foundations of global intelligence networking from technological, process and organisational perspectives and identifies the resultant opportunities and challenges. It starts by reviewing technology developments, including the capabilities and impacts of electronic networking, and in particular the Internet. It then explores how these can enhance intelligence processes. Frameworks and guidelines for integrating these intelligence processes are then proposed. Finally, the organisational challenges of embracing global intelligence networks in an effective way are discussed.
Relentless advance of technology
The ongoing price-performance improvement of 20-30% per year (more than any other major technology) in underlying computer technology shows no signs of abating. Today's personal computers are more powerful than mainframes of the 1970s, and we can extrapolate with some certainty that the desktop computer of the late 1990s will be over 10 times more powerful than today's and boast high quality multimedia and gigabytes of local storage as the norm. From a user perspective graphical user interfaces (GUIs) such as Windows, have made computers easier to use. No longer are they the province of a small minority. They are widely used and impinge on virtually all business activities at all levels of management.
Another significant trend has been the growth of networking. The timesharing and on-line systems of the 1980s have evolved into a diversity of types from local area networks (LANs), to corporate networks, and public networks accessible from phone sockets. The last few years has seen the growing practice of connecting networks to each other, known as internetworking, culminating in the internetwork of very many networks, the Internet. This interconnection has proceeded at such a pace, that it is estimated that over 30 million people can connect to each other through the Internet.
Alongside these changes, the rapidly developing (and increasingly deregulated and competitive) telecommunications infrastructure, provides the basis for the global "information superhighway", whose vast information and intellectual resources are available to be harnessed for business and personal benefit.
These developments have significantly extended the role of the computer and its impact over the last few decades. From its initial use in data processing on fairly inaccessible machines, it has extended its reach throughout and beyond the enterprise, and its scope to cover information processing and widespread communications. Figure 1 illustrates this. It shows the extension of the role of computers in two dimensions. The first dimension is that of usage, with today's usage involving work teams (even globally dispersed ones) and inter-organisational working. The second dimension is the variety of functions performed. This has expanded from processing formal structured data, to embrace many forms of data (text, image, sound) and includes the informal (such as electronic mail "conversations") as well. The next phase of this evolution of role is now becoming apparent. It is that of computer support for knowledge and group working.
Let us now consider how technology developments have affected two facets of business intelligence activities, those of information access and those of person-to-person interaction.
More powerful information processing
There have been many developments in processing text-based information and databases. The following are given as illustrative examples:
- Information retrieval software, formerly an expensive mainframe option, is now available as low cost PC software packages. Examples include IdeaList and Personal Librarian.
- Newer forms of text and document indexing and retrieval, such as natural language, concept-based retrieval or bit mapped indexing, are now available on powerful search engines, such as Verity's Topic, Infoseek or Oracle's Text Server and ConText.
- Document management systems, storing thousands of documents, that can be indexed and stored, are becoming commonplace.
- CD-ROMs, with an ever growing list of titles (over 8,000 in English) offer convenient reference information from inexpensive drives on a single PC or network.
- Searching across multiple databases; no longer are searchers confined to searching multiple databases or accessing a single host. Many commercial on-line hosts now provide this facility and systems like Verity's Topic use a distributed server architecture, where information can be stored on different types of host computer.
- "Intelligent software agents" have been developed, that can carry out a number of functions, such as roaming around distributed networks seeking specific information.
- Informal information sources, especially those on the Internet, have multiplied.
- Information "filters" can be used to screen informing information against user-defined rules, so that relevant information, such as containing certain keywords or coming from a given sources, is prioritised.
Several information service providers have exploited such capabilities by developing a number of strategic information services. There are industry specific newsletters, many delivered directly to the users desk-top. There are tailored and customised news services, such as First!, provided by Individual Inc. There are market and company reports, easily accessible through a Window's interface, by business professionals without needing help from information specialists. Reuter's Business Information Service and M.A.I.D. are examples.
While such developments have made information much more easily accessible, it requires additional sophistication to overcome the accompanying the problems of information overload. This require greater attention to proper information management, (and information resources management) a topic of great importance, but outside the scope of this article.
In retrospect, we can now see that the advent of electronic mail (email) was the start of a widespread move towards using electronic networks for more informal information exchange, rather than simply database or programme access. Computer networks are increasingly used to enrich human-to-human interaction, and as a result offers a new dimension for improving intelligence processes. Developments in this field include:
- Evolution of electronic mail from its earlier use exclusively with internal divisional or corporate users, to its growing use for external communications.1
- More public email systems, ranging from those of common carriers (e.g. MCI Mail or BT's OfficeLink) to those of network service providers, such as CompuServe, who has over 2 million subscribers world-wide.
- Growth of computer conferencing or bulletin board systems, where people sharing a common interest, can input information to "conferences" or "forums", which are structured into "notes", "topics" and "threads". Examples of such systems are Lotus Notes (popular with large corporations), First Class (popular with membership organisations, international community and environmental groups), or CoSy (as used by UK public network service provider CiX).
- Group decision support systems, such as meeting systems incorporating brainstorming, idea organising, rating alternatives and issue exploration. Examples include Ventana's GroupSystems or CTC's VisionQuest.
- Co-operative supported co-operative work (CSCW) environments enabled by "groupware", software tools that let work teams share on-screen "whiteboards", information, documents and applications.
Such capabilities make the recording and sharing of informal information easier and more widespread. Conferencing systems, in particular, make it easy for individuals to pass relevant information to those within their "network" and beyond, or to join discussion groups on topics of interest and to develop new networks for information sharing and intelligence gathering. Another important facet is that of access to a wide range of expertise, without having to decide in advance who to consult. Thus, a person with a problem or request can post it into an appropriate conference, and receive a response from people qualified to answer. Responses may emerge from unanticipated and unlikely quarters. In effect this is an "answer network", an interesting blend of computer technology with human knowledge, that performs better than either computers or humans separately.
One long-evolving development, that epitomises many of these capabilities, is that of the Internet. Only in the last year or so has it become the subject of much attention from business, but its significant rate of commercial development means that it worthy of closer study by business strategists.
The Internet - A Network of Networks
Statistics about the Internet always seem staggering. Some 3 million computers all over the world are connected to it, ranging from large corporate systems to those at individual's homes. Traffic growth almost doubles every year, and for some specific services, such as the World Wide Web, every few months. The most widely used facility on the Internet is that of electronic mail. However, the following facilities are also highly relevant to the business intelligence community:
- File transfer: documents can be transferred on an individual to individual basis, such as attachments to electronic mail, or retrieved from public or password protected repositories using ftp (file transfer protocol).
- Discussion groups or newsgroups: around the Internet are over 10,000 newsgroups to which individuals can "subscribe" and contribute. Their subjects cover virtually the whole gamut of human endeavour from activism to zoology.
- The World Wide Web (WWW): a hypermedia system where documents (that can contain images and sound as well as text) have "hot-spots" that when selected automatically route the reader to the linked documents. Such links let users literally gather information from scores of machines around the world in just minutes.
- Powerful search engines for WWW, such as CERN's World Wide Web crawler or Carnegie Melon University's Lycos, where over 3 million documents are indexed for retrieval. New search engines are added almost daily.
- Gopher: structured menus of information, long favoured by librarians for information retrieval on the Internet, along with other search tools like Archie, Veronica.
- List servers: somewhat like a newsgroup, but instead of the user having to check it regularly to find out what's new, they automatically receive an email of any new entry for those lists to which they "subscribe". One example is the UK Government's COG's (Collaborative Open Groups) on topics such as electronic commerce and access to parliamentary information.
Of the various facilities, the World Wide Web is quickly becoming the information resource of choice for the ordinary Internet user. It is also attractive to newcomers who, using software such as Netscape can be up and running within half an hour. Netscape is one of a growing class of software called a Web browser, that helps the user "navigate" around this vast pool of information and view it on their screen. Many companies and organisations are making their information available on it, either at no cost or for a subscription. In late 1994, there were 6000 sites offering information on the Web, a number reported to be doubling every three months.
The Internet illustrates both the advantages and drawbacks of a highly informal and heavily connected world of information resources, something that Xerox researcher Englebert described as a "docuverse" (document universe) of information. Advantages for users of business information are:
- Vast resources of information (being added to at tens of gigabytes a day), at no or low cost.
- Significant sources of research, social and environmental thinking (one source was regarded by Shell planners as one of the key sources of for such inputs to their future planning and scenario building).
On the negative side, putting aside any problems of network access and overload at peak times are:
- Little high value market or business information (though seems to be changing as electronic pay-as-you go mechanisms become established).
- Little formal structuring and classifying of information (though some directories, resource and "bookmark" lists are now appearing - see Appendix for some of the sources of business intelligence used by the author).
- No quality control (the Internet is often portrayed as anarchic).
- A certain amount of serendipity as to whether you stumble upon the information you are seeking.
Therefore, for formal published information many businesses will continue to prefer pay significant sums to authoritative sources, who also add value by structuring their information. However, it has to be noted that they too, are increasingly making their information available to Internet users, and the question remains as to how much their information is really worth when compared to the significant amounts of similar information (such as US public information, including corporation's annual SEC filings) available at no cost or for much smaller sums. It seems that the Internet could influence information service prices in the same way that packaged PC software did to software prices. Thus quality newsletters on the Internet are charged at less than $50 a year compared to nearly ten times that for printed equivalents.
The real significance of the Internet, though, is not so much its information capabilities, but the capabilities it offers to move beyond information to intelligence. But before looking at the role of the Internet, it is necessary to review these conversion processes.
From Information to Intelligence
Intelligence has many definitions and interpretations. For the purposes of discussion here I consider intelligence as the result of refining information through human interpretation into a form suitable for management decision and action. Intelligence is produced from four interlocking sets of core processes:
- As the foundation are those processes associated with the direct manipulation of information, throughout its life cycle. One example of an integrated set of such processes is that of information refining, taken to a high level of proficiency by Trend Monitor International. 
- Associated with these 'production' processes are development processes that keep them effective and current. These include identifying and classifying new issues, adjusting to changing user needs and sourcing.
- The next set of processes is characterised by human analysis and interpretation of information. This are complex cognitive processes where individuals and teams develop and refine their mental models of the external environment, markets, businesses, competitors, etc.
- The final set of processes is that of turning individual and collective knowledge into strategic action. This involves decision making, often in groups.
The facility of integrating these processes, such that an organisation senses its environment and responds accordingly is increasingly recognised as creating a 'learning organisation'. This term is, in my opinion, frequently abused or misused. Many people interpret its key process as individual training and not, as this model suggests, a complete set of information and knowledge processes, of which we have surfaced just a few.
Another way of considering these processes and the role of computer is to consider the framework shown in Figure 2. Here they are categorised according to the degree of understanding of a problem and whether the computer is playing an information or knowledge leverage role. Some of the ways in which computers can aid the different processes are as follows:
A) Work flow and thesaurus management. The computer can be used to automate information collection (e.g. from news feeds), and assemble it into appropriate categories and route it to individuals for interpretation.
B) This is the more sophisticated use of computers, such as concept identification, abstracting and the use of artificial intelligence techniques to find and filter information.
C) Here computers can be used to help communication between experts, or as a vehicle for discussion of refined information. Various forms of groupware can aid cooperative working on shared documents.
D) This involves the most sophisticated set of tools, many of which are still undergoing development in laboratories. Such tools may include conceptual modelling aids and knowledge support systems.
Such frameworks are a useful aid to those responsible for intelligence gathering and management. They can be used to check the overall balance of resources, to identify opportunities for investment in new technologies and improved processes, and as a checklist for benchmarking overall intelligence capabilities against those of other organisations.
Importance of Networking
Studies of successful organisations and especially those good at innovation reveal the importance of strong human networking, especially externally, such as with customers, market influences, peers.  Such organisations are also likely to have excellent intelligence systems exhibiting most, of not all, of the following characteristics:
- A strong external orientation, extending beyond the most immediate external influences to those such as customer's customers, regulatory and legal influences, social.
- Good sensing mechanisms, individuals with their "ears and eyes "close to the ground" and taps into key sources of intelligence.
- Efficiency in terms of resources used, for example by minimising duplication of information activities and maximising the effectiveness of costly human resources.
- Reusing information collected for one purpose for another higher value purpose. An example of this is the growing interest in data warehousing to provide transaction data in a form suitable for customer and product analysis.
- Effective use of a variety of tools, both computer based and procedural.
- Critical analysis and appraisal capabilities, providing ways of stretching thinking and going beyond the established unthinking consensus or 'groupthink' found too often in many organisations.
- Well developed networks - of experts within and beyond the company.
In several of these areas, the Internet and other forms of open electronic networking2 have an important part to play. For example it provides an ideal vehicle to tap into the external information world and also enhancing effective networking and simplifies access to external experts. To illustrate this in practice I will draw on two generally accepted adages about business intelligence:
- 90 per cent of the base information is in the public domain
- most of the value of intelligence is not the information but in its interpretation and resultant action
Take the first adage. The most effective way of finding available information is often not by using sophisticated computer search engines, but by asking an expert where to find it. And if they don't immediately know the answer they usually "know a person who does". That is one strength of the Internet (at work in quadrant C of figure 2). You find a newsgroup or list server of experts in a given field and ask an appropriate member or post to the groups "do you know where I can find ....". There are several advantages of humans over search engines. Not only can they give you pointers, but they can often qualify them for you, advising you on quality and relevance for your needs. Furthermore many are experts because of their own networks of sources and contacts. Such experts use the Internet to maintain and develop a larger and more competent network than they could by other means. And they may be able to get you that 10 per cent of information not in the public domain.
The second adage is exemplified by the use of computer conferencing to interpret and develop information into intelligence. A specific conference can be used to deposit collected information, but also to have individual comments. When run in a specific manner, such as during a planning or scenario building exercise, the conference leader or moderator as typically known can organise comments around evolving structures (this is quadrant D of figure 2 in action). Such an approach has the advantages over conventional methods such as commenting on documents or having face-to-face-meetings. People can contribute at times and from places convenient to them. The degree of interaction can be thoughtful (recorded and structured in computer) and rapid (several iterations in a short time) and extended (the discussion remains as "organisational memory"). The medium works very well in the time leading to important planning meetings, and has been adequately illustrated at companies like Digital who have used VAX Notes as a key planning vehicle for years.
I can illustrate the power of open electronic networking reinforcing both these adages from a recent personal example.
A client, considering a new business venture, wanted a very rapid market assessment, completed in a day, not the weeks normally allowed on a consulting assignment. Here is how our small team used some of the various computer assisted information and knowledge techniques mentioned earlier:
- Forewarned that we might be asked to do this, we used electronic mail, and observation of newsgroups to identify experts and email to check availability of 2-3 selected experts (thus improving communications effectiveness and reducing costs by avoiding telephone tag)
- We already had (as part of our market monitoring) refined information on the relevant markets and suppliers on our local PC database in semi-structured form - keyworded for subjects and market related topics with free-from text.
- We were able to search CompuServe forums, especially the information libraries, and do direct searches on publicly available databases (on a pay as you go basis, rather than the usual hefty corporate annual subscriptions).
- We used our network of experts to contribute their specialised knowledge, updated by phone and additional text searching, by transmitting pages in preset formats.
- Key parts of the draft document were emailed for comment and revision during the latter part of the day (had we all shared a common service, we might have used computer conferencing).
Finally, the client was faxed and emailed the final 20 page document at the end of the working day, all without the contributors or the client having had a meeting. Our phones lines were very busy both with voice, data and fax traffic - networking raw information, people and the resultant intelligence.
The features illustrated by this example are the use of several modes of computing in an integrated way - information refining, information access, interaction between contributors, development and refinement of intelligence through computer enhanced human interaction. It should be pointed out that the team members were at three different locations (their homes), and the client contacts were at two sites. Above all, this example, illustrates the use of computers and communication for rapid interaction both within the team and beyond, for an intelligence gathering and processing exercise.
It is the capabilities of networks such as the Internet to link experts to each other that I believe will have a more profound impact on business intelligence than all of the sophisticated tools that seek and find information from the thousands of resources scattered around the world. Intelligence processes can be made more effective from the proper management and the enhancement of human interaction. Building these into a global intelligence network is therefore about shifting the focus of current practices in business intelligence from
published databases -> informal sources
information retrieval -> information refining
computer algorithms -> intelligent agents
information processing -> human networking
in-house analysis -> networks of external experts
information databases -> information refineries
undirected briefs -> quick focused reports
information technology -> information resources
information specialist departments -> business partnerships
Such shifts require a rethink of existing processes and a melding of technology with work processes. Successful implementation requires similar ingredients to any major IT or organisational change programme e.g. a business intelligence champion, close working relationships between specialists and senior decision makers, aligning intelligence gathering with business goals, developing team work and structure.
New technologies like the Internet often create adverse reaction from established groups. For example, one high profile IT director has been cited as calling the Internet 'Micky Mouse' and 'diversionary', while line managers have described patterns of networking as 'uncontrollable'. However, just as personal computers were once regarded as 'toys' they are now widely accepted as essential working tools. So, in time, with proper focus and serious intent, 'surfers' of the Internet will become valued 'intelligence seekers'. The opportunities created by the many advances in information technology mentioned earlier in this article are too important not to integrate into business intelligence systems.
In this article I have identified how developments in information technology have helped both basic levels of information handling, but also the more complex processes of creating valued intelligence for decision making. In particular the use of electronic networking and the Internet have a major contribution to play in enhancing the essential human-to-human interactions necessary for effective intelligence gathering and interpretation. Businesses who learn how to exploit facilities such as those offered by the Internet to tap into expertise across national and company boundaries will have a key advantage over those who dismiss such capabilities as irrelevant or unproven. A framework that positions different types of computer support for intelligence processes has been described.
With global intelligence networking upon us, it is those organisations that can develop and enrich human networking, using technology as a powerful lever, that will be more able to meet the challenges of a rapidly changing environment. For many organisations the change from established information gathering and intelligence processes to these new and continually evolving globally networked ones will not be easy. However, learning to use and exploit such capabilities is a key task that must be grasped by business intelligence professionals and their business colleagues alike.
 Trend Monitor International, Jan I.C. Wyllie, AGSI Journal, pp 121-123, November 1992.
 Managing Innovation within Networks, Wim G. Biemans, Routeledge, 1992.
1. Interestingly, a 1995 study conducted for the UK Department of Trade and Industry indicates that use for external communications lags that for internal communications by a factor of 5-10.
2. By Open Electronic Networking, I am including the use of electroinc mail, computer conferencing and other communications using computer networks that are not 'closed' to a specific organisation (although the content may be defined to a closed group).
The original article had an appendix of online resources. Many of these are now defunct so the Appendix has not been updated here.
© Copyright. Infonortics, 1995. Reproduced with permission.Insight No. 9 (MkIS) - Articles List
Last updated: 19th March 2011