In addition to the requirements of science itself (see previous chapter), there is an intrinsic motivation arising from software development itself to engage with this subject: all relevant software is of such complexity that it simply exceeds the human capacity to comprehend and control it. Scientific software is no exception here, but rather a confirmation of this: the intrinsic complexity of scientific problems compels us to develop software of corresponding complexity. Without (technical) tools and knowledge of strategies developed over decades, however, this is doomed to failure.
From a software development perspective, the book is not about writing code that works (in the sense of delivering a correct result), but about code that works in a broader sense – that is, code that is, in particular, reusable, reliable and verifiable.
The motivation behind the book can be summarised in a single sentence as a provocative thesis: Code written by scientists is generally of far too poor quality to be meaningfully reused. However, this touches directly on the very foundations of science, for without traceability there can be no science and no gain in knowledge.
The good news is that it is possible to develop software that meets the requirements of science. The less good news is that this requires discipline, a structured approach, the ability to think abstractly, knowledge and understanding of the relevant tools and techniques, and the professionalism to apply and implement this knowledge effectively.
These questions are intended to encourage personal reflection on the topic, but no answers are provided.
An annotated and carefully selected list of further reading on the subject. The selection is, by its very nature, subjective.
A good book for those who wish to explore programming in greater depth and, above all, in a practical way within an academic context and in the more physics-oriented disciplines (including physical chemistry) — and who find the book lacks practical relevance:
Other books (also) tailored to Python and focusing on its use in a scientific context include (this list is by no means exhaustive):
For anyone wishing to explore Python in greater depth, “The Hitchhiker’s Guide to Python” by Reitz and Schlusser is also highly recommended:
A highly readable, personal account from the early days of computer science and programming, written by one of its key figures, is Edsger Dijkstra’s speech delivered in 1972 on the occasion of his receiving the Turing Award. In it, he uses the term ‘intellectual manageability’ and emphasises why, together with abstraction, it is central to programming.
The term ‘software engineering’ was first brought to wider attention at the NATO conference on software development. Naur and Randell authored the proceedings of this conference.
Another great classic, focusing on project management and, moreover, very well written, is Frederick Brooks’ *The Mythical Man-Month*. In it, the author describes his experiences developing the operating system for the first IBM mainframes in the 1960s and reflects candidly on what went well and what went wrong. The 1995 anniversary edition also includes, amongst other things, his equally famous essay “No Silver Bullet—Essence and Accident in Software Engineering”. The central thesis of this essay is:
There is no single development, in either technology or management technique, which by itself promises even one order-of-magnitude improvement within a decade in productivity, in reliability, in simplicity. Frederick P. Brooks: No Silver Bullet
Ian Sommerville provides a fairly comprehensive and well-structured introduction to software engineering in his book “Software Engineering” [Sommerville, 2018Sommerville, Ian (2018): Software Engineering, Pearson, Hallbergmoos].
The problem of programming’s lack of prominence in the sciences is now well known. Perhaps one of the first people to have engaged intensively with this problem and possible solutions is Greg Wilson. He founded “Software Carpentry”, a non-profit organisation dedicated to teaching programming concepts in an academic setting. One of the reasons articles are published in biology-focused journals is that bioinformatics would be inconceivable without computers. However, this does not mean that the average biologist is a better programmer than a chemist or physicist… (quite the opposite, in fact).
Ultimately, the quality of programmes has a great deal to do with the attitude of the programmers involved. See the discussion in Chapter 33 of [McConnell, 2004McConnell, Steve (2004): Code Complete. A practical handbook of software construction, Microsoft Press, Redmond, Washington] and the book *The Pragmatic Programmer* by Thomas and Hunt [Thomas, 2020Thomas, David; Hunt, Andrew (2020): The Pragmatic Programmer, Addison-Wesley, Boston].