Thursday, November 16, 2017

Autumn 2017 report to the Computer Conservation Society

This was presented on November 16 to the Computer Conservation Society by Doron Swade.

Work has continued steadily going through Babbage’s manuscript Sketchbooks (his working notebooks) and building up the cross reference database. As reported previously the volume of material is substantial and its organisation is not conducive to systematic study: it not indexed or themed by topic, nor is it rigorously chronological. Time and time again Babbage revisits the same or related topics over a period of decades, and these entries, many of which are cryptic, are dispersed through the twenty-five volumes comprising some 7-8,000 manuscript sheets. The cross-referenced database is being developed by Tim Robinson as a key research tool to manage this distributed content. 

The focus for the last six months has been on the volumes most relevant to the Analytical Engine design started in 1834. Happily this main sequence of books is, for the most part, in chronological order. We have just completed Volume 5 which covers 1841–1844, and includes the most intense period of work by Babbage on Plan 28, the most advanced design for the Analytical Engine.

There are notable gaps corresponding to periods when Babbage was travelling. There is a separate "Travelling Scribbling Book", which has not yet been studied in detail, and we know of a significant body of manuscript material in the Buxton archive at Oxford which was written while Babbage was travelling in Europe in the early 1840s. The intention is to fill these gaps in due course. 

While focussing on building the database, we have resisted the temptation analyse the material as we go as such analysis will be substantially assisted once the full corpus is in searchable form. Having said which, there are already new findings. A landmark drawing is Plan 25, dated 5 August 1840, which depicts the "Great Engine". This Plan provides the most complete system overview of the state of play at that time and is the most well-known of Babbage’s technical drawings. The version of the machine in Plan 25 is that Babbage presented in Turin in 1840 and on which Menabrea based his published description, which in turn prompted Ada Lovelace's famous Notes. The new finding is that there is no evidence of "user level" conditional control in this defining version of the design, this despite provision for sophisticated multi-way conditional branching at the "microcode" level. Given that the 1840 version was incapable of user-programmable conditional branching, the machine described in Plan 25 should be regarded technically as a calculator rather than a computer in the modern sense.

The first evidence of user-programmable conditional branching appears in an entry dated 19 March 1842 in what Babbage called the "minor operations" – ascertaining if a variable is zero, and ascertaining if a variable is + or -. This was when Babbage was well into the design of the "Small Analytical Engine" – a reduced version described in Plans 26 and 27. Work on Plan 28 (yet another complete reset) started two months later, but these “minor operations”, essential to making a "universal" machine, were carried over unchanged. It is yet to be established what suddenly triggered the addition of these operations. Babbage himself gives no hint of the reason in the Sketchbooks studied so far. 

Some 2,100 pages have so far been cross referenced, indexed, and transcribed into the database. Though this represents only about a quarter of the total manuscript material, we are much further than a quarter of the way through the process: most of the remaining volumes are more fragmentary, contain material not related to the Analytical Engine, or contain rough drafts later transferred to the drawings and/or the notations (textual content already captured). All the remaining volumes will be worked through though these are not regarded as a priority at this time.


Doron Swade

Tuesday, May 23, 2017

Spring 2017 report to the Computer Conservation Society

Doron Swade gave an update on Plan 28 to the Computer Conservation Society on May 19 as follows:

Excellent progress to report on the database for the Babbage technical archive. Tim Robinson in the US has produced a searchable database of all catalogued material with related content fully cross-referenced. Each item links directly to the corresponding Science Museum catalogue entry and to the recently available (low resolution) online scans of the originals. This work covers all technical drawings and related Notations (there are some 2,200 Notations for the Analytical Engine). A small amount of material that is known to exist but that is not yet in the Science Museum catalogue remains yet to be done.

Back in London I have completed a review of the 26 volumes of Babbage’s Scribbling Books – Babbage’s scratchpad daybooks. The Scribbling Books comprise 8,100 folio sides each of which has been examined for relevant content. The specific purpose of this review was to identify all material specifically relating to the Mechanical Notation, Babbage’s language of signs and symbols that he used to describe and specify his engines. At the same time other content relating to known unresolved questions was logged. The logs are intended as a retrieval and navigation tool to support ongoing research into the notational language with a view to further decoding the Analytical Engine designs.
It is thought to be only the third time in history that the set of Scribbling Books has been gone through in their entirety. The only known precedents for this are the work of the late Bruce Collier, and the late Allan Bromley in the late 1960s and 70s.

The next step on the main database is to systematically go through the Scribbling Books and extract all cross references, tag subject content and transcribe significant material. This process is expected to be completed by the end of the year.

Doron Swade

Friday, January 20, 2017

Winter 2016 report to the Computer Conservation Society

This was presented on January 19 to the Computer Conservation Society by Doron Swade.

The main activities since last report focus on the original manuscript sources of the Babbage technical archive held by the Science Museum. Since the 19 th century there have been several generations of reference systems, and with the digitisation of the archive in 2011 by the Science Museum, a further layer was added. The reference system now used by the Science Museum will be the de facto standard for future scholarship.

What we are doing is reviewing image by image the entire archive holdings (some 7000 manuscript folios) which include all surviving drawings and notations for the Analytical Engine, to reconcile their filenames and titles with the three or four earlier reference systems, to identify anomalies, relate the descriptions to the physical sources for untraced or missing material, identify phantom entries and omissions, and resolve situations in which material is known to exist or have existed for which no physical source is evident. Not all of this is straight forward and in some instances unscrambling layered inconsistent reference systems dating back to Babbage’s time has proved to be nightmarishly difficult. Several visits to the physical archive at Wroughton have been made. More will be required.

The outcome of this process will be a searchable data base, annotated and cross-referenced, that identifies all known sources, and records the status of each source including issues of provenance, anomalous reference history, with a record of links and citations to other material in the archive and elsewhere.

As reported in September, Tim Robinson in the US is going through all known material sheet by sheet to construct the searchable data base with estimated completion by the end of 2017. In London I am going through 20 volumes of Babbage’s ‘Scribbling Books’ page by page, transcribing AE-relevant material, to produce a searchable quick index, estimated for completion by July 2017.

We are in close collaboration with the Science Museum archivists, and our findings are routinely supplied to them in support of their ongoing work to open-source this material. The main outcome to the Analytical Engine project will be a survey-knowledge of all known material relevant to the AE design, consistent indexing, and a powerful software retrieval and reference tool.

While this activity is ongoing we have intermitted our efforts to reverse engineer a more complete understanding of the Analytical Engine design though, it must be said, pursuing the implications of critical new discoveries has proved, on occasions, irresistible.

Doron Swade

Friday, October 7, 2016

How we got to where we are

The recent update on the project reported that we have taken a strategic step away from the nuts and bolts of the mechanical drawings and are concentrating on marshalling all known archival sources that bear on the design of the Analytical Engines in its various forms.

Why we took this step had an unforeseen prompt.

Last year, 2015, marked the 200th anniversary of the birth of Augusta Ada, Countess of Lovelace, who, in 1843, wrote in collaboration with Babbage a much-debated account of the Analytical Engine. The bicentennial year focused attention on Lovelace’s life and work: symposia, colloquia, conferences, exhibitions and publications invited perspectives on the significance of Lovelace’s role. This required amongst other efforts a new and detailed look at her famous article.

A celebrated feature of Lovelace’s article was the inclusion of a description of how an Analytical Engine would automatically calculate Bernoulli numbers. The sequence of instructions, the presentational format of the procedures, and the representation of the changing internal states of the Engine have what we now recognise as program-like features. Nowhere does Lovelace or Babbage use the word ‘program’ but for convenience the procedures she described will continue to be referred to in what follows as ‘programs’ on the understanding that to do so is anachronistic.

The Crunch

Rainer Glaschick in Germany was invited to create a simulation of Lovelace’s Bernoulli example for public display in an exhibition at the Heinz Nixdorf MuseumsForum in Paderborn, Germany, and he embarked on the most detailed analysis to date of the algorithmic and instructional content of the example.

Rainer consulted with several others, Bernard Sufrin in Oxford, Tim Robinson in the US, me in the UK and others, to explore and clarify programming structure, underlying and implicit concepts, and hardware capability.

In the course of this Rainer raised searching questions about the computational requirements of the Bernoulli calculation and this prompted further questions about whether the Analytical Engine as designed by the early 1840s would have been capable of calculating the Bernoulli series as Lovelace described.

What emerged from Rainer’s analysis was a clearer understanding of Lovelace’s program. However, the question of whether or not the Analytical Engine of the early 1840s was capable of executing the calculation, and managing the results, pushed to the limits our understanding of the internal control of the Analytical Engine, specifically the question of the basic control arrangements between the punched cards of which there are several types, the processor (‘the Mill’) and memory (the ‘Store’).

Our inability to definitively answer aspects of the control arrangements was partly what prompted us to change our investigative strategy.

Why is Lovelace’s Bernoulli example so difficult?

A taster of what Rainer’s exploration raised for us was the question of memory management.

The Bernoulli numbers form a series in which each new term is generated by a repeated set of operations.

Instructions are coded on Operation Cards – punched cards that specify the instruction to be performed. Associated with each Operation Card are Variable Cards that indicate where the operand is to be found in the Store, and where in the Store a result is to be placed. The relationship between an Operation Card and its Variable Cards is ‘hardwired’ i.e. the association between them is fixed and unalterable.

The Analytical Engine could automatically wind back the set of Operation Cards and rerun them in a loop a prescribed number of times. Since the Variable Cards contain a form of absolute addressing, rerunning the same physical set of Operation Cards will place each new result in the same location in memory i.e. each new result overwrites each last result and, unless printed as they are generated, intermediate results are lost.
For the Bernoulli calculation all prior results in the series are required for the computation of each new result. So results need to be retained in separate locations in the Store for later systematic retrieval.

Lovelace’s Bernoulli program describes looped iteration that does not overwrite the results register i.e. each new result is placed in a separate incremented location in the Store. Without a form of indexed or relative addressing it is not evident how this could have been done by rerunning the same set of cards and we have found no evidence in the design drawings for a relative or indexed addressing capability. The late Allan Bromley, whose work on the designs is the most detailed and penetrating to date, surmised that this is something that Babbage did not resolve.

One way of automatically computing a recurrent series and retaining all intermediate results would be to repeat the sequence of operations by feeding in physically repeated sets of cards but associate with each set a different Variable Card to direct each new result to a new location in the Store. This would allow automatic calculation of the Bernoulli sequence. But it is clear that Lovelace wished for a general solution as a demonstration of the complete generality of the Analytical Engine’s capabilities – enter n as the number of Bernoulli numbers needed and press ‘Go’ for the automatic generation of all n Bernoulli numbers in the series using the same looped set of Operation Cards.

Between 1837 and 1840 Babbage wrote some two dozen programs at least two of which involve recurrence relationships of this kind i.e. the calculation of series requiring the repetition of the same sequence of operations to generate each next term. In the two main examples each new result is generated only from the immediately prior result, and each new result overwrites its immediate predecessor. So if the nth term of a series is required the instruction set would be repeated n times and prior results, other than the (n-1)th are not material. If the prior intermediate results are needed they can be printed as they are generated, but they do not need to be retained internally in the Store to continue the series. Such recurrence relations can be computed by rerunning the same physical set of Operations and Variable Cards and overwriting each last result with each new one.

So while Babbage had described how the Engine would deal with some recurrence relationships he provided no example of a recurrence relationship that required the internal retention and reuse of all prior results. Without a systematic form of incremental addressing it is unclear that Lovelace’s Bernoulli program could run on an Analytical Engine of the early 1840s. This is part of the reason why Lovelace’s Bernoulli example is a more challenging example than any of Babbage’s earlier examples of programs.

Being unable to definitively resolve the arrangements for memory management in a decisive response to Rainer’s questions prompted us to realign our immediate objectives and to redirect our attentions to systematically master what was knowable about the Analytical Engine design – this by establishing a comprehensive database for all known technical sources, ensure that the database searchable and, perhaps most importantly given Babbage’s sometimes fragmented way of working, to internally cross-reference the material. This is a substantial task given the size of the archive but the outcome will allow a systematic analysis and reveal to what extent existing sources constitute a coherent description of a buildable machine.

Rainer Glaschick has made public his analysis of Lovelace’s Bernoulli example. His full account, Ada Lovelace's Calculation of Bernoulli's Numbers, can be found here.

Doron Swade
October 6, 2016

Monday, October 3, 2016

Autumn 2016 report to the Computer Conservation Society

This was presented on September 22 to the Computer Conservation Society by Doron Swade.

The last report described that reverse engineering a coherent and consistent understanding of the Analytical Engine design by examining the main and best-known mechanical drawings has been less than completely successful to date. I also described that to deal with this we have taken a step back to marshal and review all known sources to assess the descriptive completeness of the surviving technical information i.e. whether Babbage’s archived technical drawings and manuscripts constitute in their entirety a coherent description of the Engine.

Tim Robinson in the US has been trawling through the entire technical archive and compiling a searchable cross-referenced data base for all surviving technical material. In parallel with this I am conducting a fast-track survey of some twenty manuscript volumes of Babbage’s notebooks focussing on material on his notational language (the Mechanical Notation) that he used to describe his machines – this with a view to reading the notational description of the AE designs using the decoded Notation as an interpretative tool to achieve a deeper understanding of the designs. The data-base and trawling exercise is what has primarily occupied us over the last four months.

A major step forward in the overall project has been the online public release by the Science Museum of the major part of the digitised archive. This has been widely anticipated and widely welcomed. The Museum’s intention is that the images are for the time being illustrative only in that the image resolution is sufficient to identify the drawing and its major features, and higher resolution versions are available on request. The current lower resolution images are sometimes adequate for smaller manuscripts but for the larger sheets the detail of the smaller annotations is insufficiently clear. The work of our small team is not hampered by this as we have access to higher resolution off-line images under licence. The relevance of the public release and the usability of the images affect the volunteer effort of folk wishing to help the project, for which they need access to detailed images. The good news is that the Science Museum intends a higher resolution release of the whole archive for early in 2017, and this will include material already digitised but not yet released online. Feedback in response to the first release has confirmed the wider interest in the project and the presence of fair number volunteers eager to help. We expect to benefit from this effort more fully in the new year when the images will support close reading.

In short, we are retrenching by focussing on identifying, reviewing and indexing all known the technical sources to serve both as a datum and as a research tool in the pursuit of a deeper understanding of the designs.


Doron Swade

Thursday, May 12, 2016

Spring 2016 report to the Computer Conservation Society

This was presented on May 11 to the Computer Conservation Society by Doron Swade.


We have been pecking away at Babbage’s original design drawings for some while now and have found with regret that we are unable to reverse engineer a coherent and consistent understanding of the Analytical Engine from the mechanical drawings alone. There are some 300 drawings and some 2200 Notations – descriptions of the mechanisms using Babbage’s language of signs and symbols. There were three phases of design - early, middle, and late.There is overlap between these, there are ad hoc upgrades, and only fragmentary explanation, where there is explanation at all.

It remains unclear whether any of these three phases is graced by a complete design. This in itself would be unfortunate but not catastrophic as mechanisms can be devised as functional replicas for omissions provided the intended function is fully understood. The immediate problem is that the extent of incompleteness is not clear. The work of the late Allan Bromley in decoding the AE designs in invaluable but he published only a small part of his substantial findings and these are anyway based on only part of the archive. While much is understood about many of the main mechanisms and the general scheme, there remain fundamental aspects control and sequencing that are not yet well understood and have resisted further illumination.

To achieve a more comprehensive understanding of the designs Tim Robinson in the US is going through the entire Babbage archive (over 7000 manuscript sheets) and producing a cross-referenced searchable data base. The purpose of this is to marshal all known sources so that we have a bounded idea of all relevant material. The intention is to reveal any explanations and/or drawings that Babbage might have left that have not yet come to light.

A second line of attack is to the 2200 Notations for the AE using the newly acquired knowledge of the Mechanical Notation. Allan Bromley maintained that the Notations were indispensible to his understanding. However he did not publish how he had used the Notations and he was the last to use them as an interpretative tool. The hope here is that the Notations will provide some of the missing information about logical control and moreover give insights into design strategy. In parallel with Tim’s comprehensive data base index I am going through the twenty volumes of Babbage’s ‘Scribbling Books’ identifying all material on the Mechanical Notation – a fast-track way of accessing this specific material.

Currently the stages envisaged for the project are:

1. Finalising research of the original design
2. Specifying a viable version
3. Computer modelling and simulation/3D printing
4. Manufacture and construction

We need more hands to the pumps and have latterly diverted some effort to fundraising. We have a 3-year plan at the end of which we expect have the requisite understanding of the designs, a platform from which to specify a viable version of an AE that is historically authentic, and to have trialled tools for modelling and simulation. The funding proposal includes pulling in requisite expertise including modellers and mechanical engineers.

In summary, we have had to bite the bullet with the realisation that without a concerted assault on the sources, a fuller understanding of the Engine design will not be forthcoming. We need to understand the intentions of the design well enough to identify missing mechanisms and understand their intended purpose well enough to devise fill-ins that are consistent with Babbage’s design style. Effort is now divided between continued study of the designs and fundraising to ramp up the effort.

Doron Swade

Tuesday, May 3, 2016

Winter 2015: Project Report to the Computer Conservation Society

The following report appeared in the Winter 2015 Resurrection:

The bicentennial year of Ada Lovelace’s birth falls this year. Preparation for various celebratory events has directed attention to Ada Lovelace’s ‘program’ (1843) to calculate Bernoulli numbers using Charles Babbage’s unbuilt Analytical Engine. A small group including Tim Robinson in the US, Rainer Glaschick in Germany, Bernard Sufrin in the UK (and me) have been collaborating in exploring the ‘program’. Significant progress has been made with many obscurities now illuminated. The study has directed new attention to how the several types of punched card control the internal microprograms on the one hand, and how these functions interface with the user on the other.

There has been significant archive activity. The major historical source is the Babbage technical archive held by the Science Museum. The Science Museum digitised the archive in 2012 is now preparing to provide open access to the archive. The Analytical Engine project team has been the main user of the digitised archive under special licence. In the course of the project mismatches have been identified between the digitised material and the existing printed index compiled by the late Allan Bromley and published by the Science Museum in 1991. Referencing anomalies, identification of material omitted from digitisation exercise and other structural issues have become evident. Descriptions of these have been compiled and we are working with Science Museum archivists to resolve and correct these ahead of open access release. The work is detailed and, given the volume of material, substantial. Eye-strain is an ongoing hazard.

This archive work has suggested a new and significant prospect for the role of the Notation in an understanding the designs. One of the difficulties in understanding the designs is the need to reverse engineer logical function from mechanical drawings of mechanisms - this without textual explanation of purpose or intention. The original hope was that the notations, expressed in Babbage’s symbolic descriptive language, would contain a higher-level logical description that would relieve this difficulty. As described in earlier reports the main features of the Notation were decoded from a detailed knowledge of the mechanisms of Difference Engine No.2. The provisional conclusion from that study was that the notations are a description of the mutual physical relationships of mechanical parts and are not an abstraction of a logical description of the Engine’s function. Further, that the mechanical design preceded the notational description. New material found in the archive suggests that while this might be true for Difference Engine No.2 the notations for the Analytical Engine (about 2,700 of these) may indeed embody higher-level logic, control functions for sequencing the punched cards and orchestrating the internal operations that the punched cards control. If so, the notations would provide the explanatory tool we have been looking for and the prospect of this is enough to distract one from the plight of refugees migrants and austerity, even if only briefly.


Doron Swade