Matt Willard

The Pragmatic Programmer 20th Anniversary Edition

BE PRAGMATIC: flexible, results-oriented, adaptable based on the task
PRAGMATIC PROGRAMMER QUALITIES: early adopter/fast adapter, inquisitive, critical thinker who gets the facts first, realistic, jack opf all trades

A PRAGMATIC PHILOSOPHY
-take charge/responsibility of self and career
-if commitment is made, take responsiblility, be held accountable
-have a backup plan and try/provide options; what can be done to salvage?
-clean uyp bad designs/wrong decisions/poor code soon to decrease the chance of the software breaking down swiftly
-remember the big picture, but also, be a catalyst for implementing change
-GOOD ENOUGH: what’s the requirement for good enough? When should you know when to stop? What is the accepted scope and quality for a launch now? What would the users be good with?

KNOWLEDGE PORTFOLIO:
-make it a habit to keep learning
-diversify and learn the ins and outs of a variety of technologies to be adjustable
-keep reviewing your knowledge
-critically analyze what you read and hear
–ask the five Whys
–who does this benefit (follow the money) and what’s the context

SUGGESTED GOALS:
-one new language every year
-technical book each quarter + other nontechnical books
-experiment with different tools and environments
-stay current with trade journals and blogs and such

GOOD COMMUNICATION:
know and speak to your audiennce; gather feedback, improve knowledge
know what you want to say; outline
have style match your audience but make it look good too
involve them; be a listener
keep code and documentation together

A PRAGMATIC APPROACH
GOOD DESIGN
-good design: easier to change than bad design: decoupling, single responsibility, naming, etc.
-try to anticipate with notes/recplabcale parts

DON’T REPEAT YOURSELF
Every piece of knowledge must have a single, unambiguous, authoritative representation within a system.
DRY is about the duplication of knowledge, of intent. It’s about expressing the same thing in two different places, possibly in two totally different ways.
–doesn’t count if it’s two smiliar functions with diffetent knowledge
–if you’re changing code, db, documentation, schema in multiple places; it’s not DRY
–basically you also want to shoot for creating stuff that’s easy to reuse without hassle

ORTHOGONALITY
-independent units; changing one doesn’t affect the other parts
–components should be self-contained, independent, with a single purpose:
—-faster to develop and test
—-easy to reuse/reconfigure
—-reduce risk, make antifragile
—-modular, layered, abstract design: can reduce what’s changed if something else above changes
–careful when using other libraries/things you don’t fully control
–make sure code is decoupled from relying on implementations of other modules
–avoid global data
–avoid similar functions

REVERSIBILITY
-focus on building flexible, adaptable software, since new decisions are inevitable, nothing is final and cast in stone
-should be flexible in terms of what it’s built for and how it’s deployed
-seems like a good way to do it is to hide things like third-party APIs behind your own abstract layers and to break things down into components

TRACER BULLET DEVELOPMENT:
tracer code basically lets you set up something lean that’s working fast, and then iterate and add onto it piece by piece, feature by feature
-think of making like a minimum viable product, getting a super basic structure down and then adding on top of the tracer code
-like starting with a hello world, then adding on top of it incrementally; get something that can do one use case out of the box, one feature as a whole
-users see something early, developers get started and build a structure and get use cases handled
-it also lets you find problems and adjust your aim to hit the target
-ready, fire, aim, fire, aim, fire…
-propotying’s exploratory and can be junked - tracer dev code goes into the final product

SPEAKING OF PROTOTYPES:
-quick and easy: drawing of a UI, or a mockup, post-it notes, etc.
use them to learn and refine fast; toss them when you’re done, tracer dev is more reliable if you need an actual framework that works

THE BASIC TOOLS
USING PLAIN TEXT
-plain text is great for keeping knowledge and easy testing
-plain text is a standard everything can run on

USING SHELL
-command shells grant you a ton more productivity and power than just GUIs can do; automation, admin commands, etc.
-can even set macros/aliases for a lot of commands

USING YOUR EDITOR
-stick with one editor: code, docs, notes, etc.
-find and interalize good shortcuts and commands
-see what else you can extend and bring into your editor

VERSION CONTROL
-remember that version control is working on different branches and versions of those branches so you can rewind when you need to
-it also tracks changes, lets two people work on the same thing, and then the system can handle the merging
-ALWAYS USE VERSION CONTROL: even if you’re by yourself, even if you’re writing a book

DEBUGGING GUIDELINES
-just another form of problem solving
-get all the data and reproduce the issue to understand; do testing
-use a failing test top isolate
-divide and conquer on the error to see where the step lies
-check your logs
-don’t make assumptions - prove them

TEXT MANIPULATION
use a text manipulation language like Python/Ruby that can work with text

KEEP AN ENGINEERING DAYBOOK
-keep a dated daybook to record what was done, what was learned, ideas, etc.
-more reliable than memory
-good for breaking down problems: rubber ducking on paper

PRAGMATIC PARANOIA
Accept that you can’t write perfect software. Thus, code defensively to doubt and validate all information, using assertions to verify. Play it safe.

DESIGN BY CONTRACT
This means documenting and agreeing on what a software module claims to do, and nothing more or less.
=> If all the routine’s preconditions are met by the caller, the routine shall guarantee that all postconditions and invariants will be true when it completes. Be strict in what you’ll accept, promise as little as possible in return.

A class invariant is basically a rule that must be obeyed from the start of program execution to finish. This can be compared to state, in that you pass state to functions and get updated state as a result. When you design by contract, you can use runtime assertions to make sure everything lines up in the contract. You can validate pre/postconditions and invariants and then do early crashes for accurate information.

DEAD PROGRAMS TELL NO LIES
It’s best to let programs crash as early as possible, especially if they run into an “impossible” error that can “never happen”. Don’t write a bunch of excess exceptions to a method, the system automatically handles that in most scenarios. Exit programs as early as possible if something goes awry.

ASSERTIVE PROGRAMMING
Don’t fall for the trap of thinking something can “never happen”, because it probably can, and probably might. Add code to check and assert it. However, use them for things that should never usually happen, not for common error handling. You should also keep them on for the most part when shipping into production, because testing can never fully account for what might happen in a production environment.

HOW TO BALANCE RESOURCES
Make sure that routines that allocate resources are also the ones to close it, ideally in the same place like within the method. You can also scope these resources to their own enclosed blocks to make sure they get closed when they leave scope. (If working with objects, you can wrap resources in classes that get cleaned up by the garbage collector later.)

Make sure to deallocate in the opposite in which you allocate them. Also, allocate resources in the same order if it’s needed in multiple places.

try/catch/finally blocks are good for handling deallocation where exceptions are involved. (Allocate FIRST, then handle the rest in the block.)

Use wrappers for resources to keep track of allocation/deallocations, and check that resources maintain a certain state.

DON’T OUTRUN YOUR HEADLIGHTS
Always take small steps, check for feedback, and adjust before moving on. This cane be covered by user feedback, test results, etc. This applies to planning as well: when you estimate months into the future or design for future needs or maintenance, that’s too big of a task.

Design your code to be replacable in case something better comes along.

BEND, OR BREAK
Code should be written as loose and flexible as possible to account for a constantly changing world.

DECOUPLING
Software needs to change, so the less tightly linked things are in parallel, the better. Indvidiual components should be cpupled to as few other components as possible. There’s a few ways to address this.

  1. Shrink chains of method calls. When you write something like:
    totals = customer.orders.find(order_id).getTotals(); Then it makes it harder to change things if you need to add something to the middle of this chain, because it assumes that all of these linked objects have relevant methods to support what it’s trying to do.

TELL, NOT ASK: Instead of “can I have a list of orders for me to search?” it should be more like “give me this order with this ID directly”

When you decouple responsibility to its own objects, you can reduce the number of calls. (This is a little hard to explain in text without getting direct experience.) But essentially you’re trying to set it up where each object (customer, orders, etc.) has its own set of responsibilities and what it knows and doesn’t know.

  1. Avoid global data. Global data coupled with methods is heck to maintain and update. Even if you put all your global data inside of a singleton object or module, that’s still global and that’s still coupled. You’re better off wrapping this data with an API that serves as an interface to pull this data.

Keep code shy: only have it deal with things it directly knows about.

JUGGLING THE REAL WORLD
Apps in the real world need to be responsive to what happens in that outside world, responding to events and adjusting based on them. There are four strategies that help with this.

  1. Finite State Machines
    A state machine is a set of states with defined events/transitions for them. Basically, the app has a starting state. Based on the event it gets, it’ll switch to another state, and anticipate certain transitions that lead to other states. (“awake” to “sleeping” via a “sleep mode” event, for example.) If we get events we DON’T anticipate, we can go to an error state. Transitions can also trigger other actions.

Here’s a simple example: a video game character has states. If they’re in the isJumping state, then they can’t transition to certain actions (like jumping again) and can only go to other states (falling, dive attack, etc.)

  1. Observer Pattern
    Clients called observers listen for events (that are observable). The observable is registered with these and when the event fires, it calls each registered observer. This is a good working pattern but it innately introduces coupling and performance issues.

  2. Publish/Subscribe
    This is a version of the observer pattern. Basically, a publisher writes events to named channels, and subscribers subscribe to these channels to listen and fire stuff off. This helps decouple the code.

  3. Reactive Programming and Streams
    A stream is essentially a collection of events, and can be manipulated on the fly like other data collections (not a 1-to-1 example but close enough for jazz). This allows reactive programming: when one value changes, and another value refers to that first value, this second value then changes to update. Since streams can then react to stuff as they arrive, this allows async operations, and is a good thing to team up with the publish/subscribe pattern.

TRANSFORMING PROGRAMMING
Programs transform input data into output data. When you think about and structure programs this way, the problems of OOP details, coupling, etc. go way down. Think of it like an assembly line.

One way to find transformations is to go top-down: start with your requirement, define inputs/outputs, then find the steps in between. These steps transform one style of data to another and these individual steps can also be transformations. You could very easily have a bigger function comprised of smaller functions, each one with their own transformation steps.

The cool thing about this is that you deal with much less OOP hassle, because now it’s a flow all in one organized line with decoupled, reusable functions, not tied to classes. Code becomes cleaner, design becomes flatter.

Some languages more easily support transformations than others (F#, Elixir, Swift, Clojure, R, Haskell). But even without innate functionality for it, you can still do const x = function1(), y = function2(x), z = function3(y).

For error handling, wrap these values in data structures/types to validate. You can handle it inside or outside of the transformations, and there are a couple of ways to do this with function calls or function values.

INHERITANCE TAX
You really don’t want to be using inheritance because it introduces problems. It promotes coupling, from parent to child to everything in between and before/after. Changes to what’s above can break what’s below. They can also get very bloated and complex with unneeded methods available, even though many languages these days only allow single inheritance.

There are two techniques to use instead of inheritance:

  1. Interfaces
    Remember that when you give a class an interface, it has behaviors the class must implement (if a game Entity gets Controllable, then it must implement methods for controlling it with game input, as an example.) This creates safety because you know classes with interfaces MUST have their methods. This promotes polymorphism without the coupling of inheritance, and greatly reduces the baggage of unneeded methods from the parent classes.

  2. Mixins
    A mixin is when you make a set of functions and extend a class or object with them. This allows you to really specialize based on what you need and only add what’s required to the specific classes or objects, for specific functions or situations. This allows you to share functionality with far less coupling.

CONFIGURATION
Put simply: keep config information outside of the app. Do this with environmental variables (.env files) or, even better, wrapping it in a thin API. An API is good for quicker updates and changes based on operating environment, without rebuilding the code.

WHILE YOU ARE CODING
LISTEN TO YOUR LIZARD BRAIN
We’re ultimately primal creatures that have learned how to be sophisticated, but lizard brain is still ticking, looking to protect us and guide us instinctively. It’s okay to be aware of it every once in a while. (Sounds like a good reason to meditate.) But when it pops up while you’re coding, it might have a few good reasons:

  • Sometimes it’s just fear.
  • Sometimes it’s aware of something instictively wrong that your rational brain needs time to realize.
  • Sometimes things are harder than they should be and that’s a signal of another problem.

One of the best things to do is give your brain time to cook on a problem - go do something mindless away from the code. Go walk, eat, talk to someone, sleep, etc. Rubber ducking is often the result.

If you’re even more stuck, another thing you can do is prototype for fun. Try prototyping something similar to what you’re working on in another window or small project, knowing that there’s no stakes and you can throw away your prototype when it’s done. This can also help your brain to cook on a problem.

Basically, don’t dismiss your instincts or gut feelings entirely, it might be trying to tell you something important. Don’t suppress, observe.

PROGRAMMING BY COINCIDENCE
Programming by coincidence is when you end up relying on luck and circumstance as opposed to deliberate, thoughtful programming that tests your assumptions and leaves nothing to chance. Things that “work” might not actually work, or work based on undcoumented boundary behavior, bad calls, etc. (This is why designing by contract is helpful.) Humans are flawed - we see patterns that don’t actually exist, and we make assumptions that aren’t true.

To program more deliberately:

  • Bash out a plan and work from that.
  • Document your assumptions and test them. Depend only on what’s reliable.
  • Make sure you can explain the code simply, and know why it works, so you can know why it can fail.

ALGORITHM SPEED
Big-O notation is the standard method of approximating the speed, impact, and resource usage of an algorithm. There are a variety of notations (beyond the scope of these book notes) but many can be figured out with some common sense. Typically, loops (especially nested ones) and recursion tend to seriously affect the completion time of an algorithm. While a lot of good, optimized sort functions exist that you don’t have to reinvent, vary your input and chart it to get an idea of how long your algo takes based on certain amounts of input.

REFACTORING
Refactoring code is gardening, weeding, pruning: an activity of regular, small maintenance for a codebase. You should be refactoring as you go, making small improvements over time right away to improve the design, performance, and general shape of the code. It’s especially best to refactor when the problem’s small, and to do it a lot, before it spirals out of control.

Keep this in mind:

  • Make sure the tests are good and they pass, and run them regularly while refactoring.
  • Refactor OR add functionality, don’t do both at the same time.
  • Take short and careful steps, and keep testing to make sure things are still working.

TEST TO CODE
Test-driven development - the art of using tests to develop code - is more important and more widely practiced now than it used to be. There’s a great insight here when they say “a test is the first user of your code”. It gets you thinking about HOW to test your functions, allowing you to reduce compiling, increase flexibility, and gather feedback right away. Decoupling helps make your code way more testable.

Don’t be a slave to TDD however - keep an eye on the big picture. Don’t go too hog-wild on a ton of redundant tests. (One school of thought is: write tests, not too many, mostly integration.) A good way to keep overall design in mind is to build end-to-end: small E2E pieces of functionality, with layers added onto over time.

You can’t be seduced by the glow of functioning tests over actually hooking the software together with a destination in mind. Testing is still important however, both before deployment and during. Unit testing is a good way to do this, because at the end of the line, software should be more like seperate testable modules chained together to work. It’s good to test as you’re building, plus test in the field - insist on building a culture of testing with clean, robust code.

PROPERTY-BASED TESTING
Good testing also has contracts like other code: meet the conditions when you feed input, and get a certain guarantee about the output. It also has invariants (things that remain true about a piece of state when passed through a function). Together, this is a property, and it can be used to automate testing.

When you’ve used a test suite, you’ve probably provided set data to perform operations on and verify. With property-based testing, you use a bunch of arbitrary data generated according to some specification and performs operations on them, checking if some guarantees are met no matter what style of data. Haskell’s Quickcheck and Python’s Hypothesis are libraries that aid this.

With property-based testing, you can uncover assumptions that you might not have thought about. You can take failed parameters in a property test and create a seperate unit test to anaylze what exactly failed and why (creates good redundancy). It’s also a great way to test and improve your overall design to account for edge cases and think about what must not change and what must be true.

STAY SAFE OUT THERE
Here’s a list of things you really, really ought to consider to protect your code from various attack vectors in a hyper-connected world:

  1. Minimize Attack Surface Area
    Keep code as simple as possible.
    Sanitize external input data before using it to prevent problems.
    Focus on authenticated services only and keep authorized users at absolutel minimum.
    Truncate risky data that might be outputted.
    Keep testing and debugging info private - don’t dump it externally.

  2. Principle of Least Privilege
    Only use the least amount of needed privilege for the shortest possible time to complete the job.

  3. Secure Defaults
    Make sure default settings are secure, even if they’re not convenient for a user.

  4. Encrypt Sensitive Data
    Encrypt sensitive data, and definitely don’t put it in plain next.
    Don’t check in senesitive data, secrets, keys, passwords, etc. into version control

  5. Maintain Security Updates
    Update patches regularly and maintain security updates swiftly.

And for the love of God, use someone else’s highly tested, reliable, long-term encryption services instead of making one up yourself.

NAMING THINGS
Name things according to the role they play in your code - don’t use “user” when you mean “customer”.

Be consistent across the entire project/team with your naming scheme and jargon.

Rename right away, when needed.

BEFORE THE PROJECT
THE REQUIREMENTS PIT
Programmers help people understand what they want in order to discover requirements for software. A statement of need is not absolute - programmers should ask questions, explore edge cases, and get in the client’s headspace. Gathering requirements is also a process, refined through continued feedback through short iterations, through mockups and prototypes to get what the client truly means. Another good way to grok what a client wants is to work as if you’re the client or user to get a real view of what’s going on.

Things to remember:

  • Program for a general case in order to support business policies.
  • Prototypes help clients illustrate HOW they want to do something more than just WHAT they want.
  • Big bulky requirements documents aren’t ideal, intended more for developers than clients. Instead, focus on breaking them down into user stories. This will also help you evaluate feature scope.
  • Maintain a project glossary for consistency between developers and users.

SOLVING IMPOSSIBLE PUZZLES
Some software problems are tricky enough where the obvious ways won’t work. the trick, beyond thinking outside the box, is to find the box. What are the constraints, and how flexible are those constraints, really?

Think of all the possible paths before you. don’t dismiss any of them. Can you PROVE a certain path can’t be taken? Prove it 100%, for sure? Categorize and prioritize those constraints.

If you need to, take a good break and let your brain cook on it.

WORKING TOGETHER
Work together and closely with users. Ideally, you want to ask experts questions and have discussions while you’re actually coding. It can be more than just pair programming - you can even have three or more people collab together as a mob. The mob typically involves people that aren’t users, too. Start small and work up - try to understand each other, and stick with small mobs and pairs until you get used to it.

THE ESSENCE OF AGILITY
Agile is how you do things. It’s not a catch-all, off-the-shelf solution. The agile “process” isn’t intended to be static, it’s to adapt to change in an unpredictable world. In essence:

  1. Work out where you are.
  2. Make the smallest meaningful step towards where you want to be.
  3. Evaluate where you end up, and fix anything you broke.

From small fucntions to large systems, continous adaptation and feedback is the way. Good design helps make agile principles easier to enact and adapt.

PRAGMATIC PROJECTS
PRAGMATIC TEAMS
Pragmatic teams are under 10-12 members. They’re small and stable - people know each other and can rely on each other, and in the long-term, have better output than constantly shifting, bloated teams.

A pragmatic team:

  • takes responsibility for fixing broken windows and maintaining high quality.
  • They monitor the whole environment for changes such as new requiremenets, increased scope, features, etc.
  • They work on things besides new features, such as systems maintenance, improving existing processes, and advancing their skills.
  • they communicate as a whole with its own identity.
  • they get answers fast because friction to communicate is low.
  • they focus on building tracer bullets (small incremenetal bits of end-to-end code) with feedback from each other.
  • they know how to automate tasks and tools.

COCONUTS DON’T CUT IT
Do what works, not what’s fashionable. (IE, maybe you shouldn’t copy Google’s dev techniques blindly, probably because you aren’t Google at the size and scope they are.) Don’t do agile just because agile is the “thing to do”. Test-pilot it with a small team - keep what works, toss the rest. You need to be flexible, based on the needs of your organization.

Deliver when users need it. Your goal isn’t do agile/scrum/lean/kanban/whatever, your goal is to deliver software in a timely manner. Don’t overinvest in one methodology and be unable to adapt.

PRAGMATIC STARTER KIT
Version control, regression testing, full automation. These are the three legs that support every project and are critical.

  • Version Control: use it to drive builds, tests, and releases. This makes a true form of continuous delivery, especially when tied to building to the cloud.
  • Regression Testing: test ruthlessly and extensively. Early, often, automatically. Test as soon as you have code. You’re not done until all the tests run and pass. Automation helps you cover and run all tests, especially before deployment. Make sure you have unit tests, integration tests, data validation, user data verification, and performance testing. Test for state coverage - property-based testing covers this. And if a bug slips through, add a new test to get it next time.
  • Full Automation: Use scripts or tools to fully automate running tests and procedures. This also applies to computer setup. Manual steps increase the chance of something messing up based on the computer and the user. Build automatically, test automatically, deploy automatically.

DELIGHT YOUR USERS
Developers are here to delight users and solve their problems. Ask this of them: “how will you know that we’ve all been successful after this project is done?” This allows you to uncover a user’s true expectations and compare requirements against them.

PRIDE AND PREJUDICE
Sign your work. If you’re responsible for it, make sure you do a good job you can be proud of, so when others interact with it, they know they’re being exposed to quality code.