I doubt very much that the inaccurate data to bolster the case for the war was an aberration. I suggest that the whole system of intelligence-gathering is all too often prone to producing inadequate, unreliable and distorted assessments, often at considerable cost. But only very rarely, as in the Hutton inquiry, is intelligence material subject to the same scrutiny, verification and testing as information governments receive from other sources. Naturally there are, by definition, genuine secrets in obtaining such material, but the whole process is wrapped around in an unnecessary aura of secrecy, mystery and danger that prevents those from outside the security services applying normal and rigorous judgments on what they produce.

It is difficult to see why, for example, Sir Richard Dearlove, chief of MI6, should have given his evidence to the Hutton inquiry by telephone. Everyone knows his name and what he does. Even his head office is probably not far behind the Houses of Parliament in its recognition factor. Yet the manner of his appearance merely enhances his mystique rather than protects national security.

As a diplomat who worked in nine overseas posts over 36 years, I saw quite a lot of MI6 at work. They were represented in almost all of those diplomatic missions. They presented themselves as normal career diplomats, but often, indeed usually, they were a breed apart. And it normally only took the local British community a few weeks to spot them. "That's one of your spies," they would say at an embassy social function. "Spies, what spies?" we would reply. "You've been watching too much television." But they were usually spot on.

In one capital, the MI6 officers rarely wore suits to the office while the rest of us did. "Why?" we asked. "Because we would stand out when we go outside the capital to meet our contacts," they would reply. Maybe they scarcely noticed that they already stood out pretty distinctly in the city. If the local expatriates could identify them in weeks, it presumably only took hours for a hostile intelligence service to spot them, even if they did not know them by name already.

The role of MI6 officers was to develop contacts - often, but not exclusively, in the local government machine - who would feed them information. This intelligence was usually paid for in cash. No ambassador (and I think very few outside MI6) knew how much the informers were paid. The MI6 station within an embassy operates on a budget that is quite separate from and kept secret from the rest of the diplomatic staff, including the ambassador. But the sums appeared to be generous and in cash. I certainly heard more than one reference to the payment of fees at a British boarding school directly by MI6 for the children of regular informants. Those agents, dependent on that money, inevitably had a strong temptation to embellish their reports to make them more valuable. And since those informants were being disloyal to their own country or employers, their trustworthiness and credibility in pursuing our interests was not always obvious.

The local MI6 officers also had an incentive to play up the importance or reliability of the sources on which they based their dispatches. Ambassadors and diplomatic staff saw and commented on their intelligence reports that went to their HQ in London, but could not alter nor stop them, nor know the identity of the sources.

The security services use phrases all the time like "an established and reliable line of reporting", or "a trusted and reliable source", or "a source close to the president", or "a source with regular access to cabinet ministers". John Scarlett, chairman of the joint intelligence committee (JIC) and a former MI6 head of station, described to the Hutton inquiry the sources of the 45-minute capability claim in these terms. So has Mr Blair to the House of Commons. But neither ambassadors who sit close to these sources and often might know them, nor the recipients of their reports, namely officials and ministers, are normally informed about their identity. It would make a huge difference in assessing the value of a report from, say, "a source close to the president" to know whether that source is the vice-president, or a household servant, or someone with whom the president lunches occasionally.

On one occasion the British ambassador, in an embassy in which I was his deputy, threw down in front of me a secret CX report [a report describing raw intelligence] that an MI6 officer had sent to London. He asked me if its contents looked familiar. I said they did, but I could not think why. "That's why," he said, handing me an article from the previous day's local newspaper. The two were practically identical in wording and in content.

The MI6 officer was called in and at first denied the link, until eventually he admitted that his position was untenable, despite having attributed his report to a "well-placed source". The ambassador said he had no objection to reports being based on local newspaper articles. "It is normal in all parts of the embassy," he remarked. But such reports from the rest of us were normally sent unclassified and the newspaper was identified. Why, he asked, had the MI6 representative claimed a different source and given it a high-security classification? "Because," came the reply, "if it were sent unclassified, other people would know what we were interested in."

That was an extreme example of intelligence reporting of marginal value being dressed up - or "sexed up" to use the current vernacular - as more than it was. But it was common to see MI6 reports on their channel that amounted to little more than gossip and tittle-tattle that the political and economic sections of the embassy would not have thought worth reporting. These "intelligence facts" were frequently so at variance with known facts that we knew them to have little or no credibility.

None of which is to say that the security services provide no value. Clearly they do. A lot of dedicated and usually capable people supply what is sometimes valuable material not available elsewhere. But among the wheat there is usually a lot of chaff, and the working methods of the intelligence agencies make it very difficult to know one from the other.

Therefore, to base major policy decisions which could cost lives upon their reporting is very high risk. A report dressed up in a CX jacket and bearing a high-security classification can easily take on an importance and a gravitas that it does not deserve. I doubt whether anyone at 10 Downing Street, on first seeing the intelligence report of the capability of Saddam Hussein to launch weapons of mass destruction in 45 minutes, asked the following questions: "Who exactly reported that claim? What had he been paid for that information? What was his rank or access? And what precisely had he revealed before that made him such a reliable source?" And if the replies were plausible, it would still have been prudent for Mr Blair to demand further supporting evidence before the information was used. I suspect that the security services would have replied, even if asked those questions, that they were satisfied with the reliability of the source and that on a "need-to-know basis", and with heavy hints of lives being at risk, their answers should be taken on trust.

The other main source of intelligence material is not much better: the electronic tapping of telephone conversations and radio intercepts collected and distributed by GCHQ in Cheltenham. The problem with their material is that the sheer volume and the failure very often to identify a source with accuracy means that its value is also often difficult to assess. Working twice in London on Foreign Office desks dealing with countries at war, I saw a flood of intercepts which retrospectively quite often accurately forecast what was about to happen. But since there were countless other intercepted reports that predicted events wrongly, it was virtually impossible to choose in advance the accurate from the false. Moreover, intercepts were usually fairly random and rarely worked when planned.

In one capital, in "country A", we were tasked to conduct negotiations with a government team from neighbouring "country B". The talks took place in country A because country B was less accessible and we had no secure communications there. The security services officer on our embassy staff was quite excited about his strategy: he said they would intercept all the other side's communications back to their capital and we would then know all about their negotiating positions in advance. Wonderful, we said. The negotiations, which were about a sensitive military treaty, went on for many months. We met the other side every week, giving them time to seek new instructions at every stage. But there was not a word from the security services officer. Then one day he announced with no little glee in his eye that he had something for us. He said they had discovered an intercept which had been sent by country B's negotiators back to their capital. But when it was decoded and translated it turned out to be the bare text, without a shred of comment or interpretation, of a single paragraph which our own side had tabled at a recent meeting and which I myself had drafted! It was the only intercept relating to those negotiations we ever received from the security services.

Information received from all intelligence agencies needs to be vigorously tested and challenged. That, of course, is a task for the JIC, which is comprised of representatives from relevant government departments but is headed and heavily influenced by the security services. This, in my judgment, is not a sufficient vehicle for rigorously scrutinising intelligence data. Ministers and parliament need to look harder and much more closely at the security services themselves and their methodology, and then use their information sparingly and selectively.

It is no small error for the intelligence community to convince a government that there is solid evidence of weapons of mass destruction when it becomes increasingly apparent that they do not exist and probably have not existed for some considerable time.