Navigate / search

My 5 Biggest Mistakes

You can’t work in this business for very long before the hope, idealism and intellectual curiosity is beaten out of you and replaced with TPS Sheets, and 15 different tools for telling your colleagues how to configure IIS so that your app will actually run on their machine.

If you’re new to this business there may still be time to save yourself. Go drive a truck, or learn a bit about your city and become a tour guide. On the off chance that you are determined to stay in software development here are my top 5 mistakes, maybe you can avoid some of them.

5. Thinking that things I am unfamiliar with are Bad
Us software developers generally have a well cultivated sense of your own ability to reason about things without actually trying them. We are professional generalisers after all.

“I don’t need to install Linux to know it’s shit”

“LINQ is unreadable”

“No, I don’t use ‘var’ I need to know my types”

At one point I had very strong arguments against var. A colleague tried to make me see the light but couldn’t. Then one day, Eureka, I got it. It was never about ‘var’. In my head the debate was “Save a few keystrokes, at the cost of not having the types in your face”. I thought I needed the types and saving keystrokes wasn’t a priority.

What I didn’t get was that getting rid of the in your face types wasn’t the price, it was the point. The saved keystrokes were incidental. It was never about keystrokes, it was about abstraction. In the same way LINQ wasn’t about writing LESS code, it was about writing code at a higher level of abstraction.

This is different from the static vs dynamic types debate which is a whole other issue, but I know now that I don’t know enough to have a very strong opinion on dynamic types.

Before you reject something as useless or bad, make sure you get the point. Your preconceptions about why a technology exists might be a million miles from the reality.

4. Thinking that things I am familiar with are Good
This is the flip side of the point above but in many ways it’s more damaging. We have a tendency to think the things we are familiar with are the solution to every problem. You can waste a lot of effort and creativity trying to figure out how to press flowers with a hammer because a hammer is what you know.

3. Thinking there’s a right way
When I started building software I was convinced there was a right way to do it. If I could figure it out I could carve out a nice career following my process.

There is no right way. Give up on finding the best way to build software, or even dividing processes, tools, techniques into Good/Bad. It’s wasted effort. One persons good technique will sink another person’s project. One person’s horrible hack will save another person’s project.

2. Thinking that I could learn software development from books
I spent wasted so much money on books. I recently threw a lot of them away. In my defense when I was learning this business the internet was in it’s infancy and there wasn’t a huge amount of curated material out there. Books were how it was done.

Even still, I bought far too many.

I did read a lot of them and I learned a lot from them but I didn’t learn how to develop software. I learned some ideas that could be applied. I could have saved a lot of money by not buying and a lot of time by not reading quite so many books. I could have spent that time actually building more software and more varied software.

Blogs, Screencasts, Online Training are the new Books. I think there’s a danger of replicating similar mistakes, spending too long thinking about software development as a theoretical pursuit, and leaving too little time to build stuff.

You’re making this mistake right now. You think that maybe there’s some insight here that will make you a better software developer. There probably isn’t. Go write some code.

Hey, at least you didn’t spend 40 $/£/€ to be here.

1. Thinking it all someone else’s fault.
If you work as a software developer you will work on a lot of teams and unless you are really lucky, most of them will be terrible. At least they will be terrible from your perspective.

Most software development is terrible. It’s frustrating, it’s unreliable. It’s difficult. It’s tedious.

It’s easy to assume that this is all someone else’s fault. There are solutions if people would just adopt them.

There really aren’t. Everyone including you is trying to figure out how to build software and make things less frustrating. The problem is that one person’s solution is the very thing that frustrates the hell out of you. And it works both ways. Do you have any idea how frustrating it must be for a project manager to have a coder tell them that estimates are useless.

I’m fully conscious of how frustrating it must be to work with me.

I don’t have any magic answer on this point other than to realize that none of us really have THE ANSWERS. It’s probably better to have a team full of people who know that then a team full of people with their own ideas, all of whom think they are right.

Of course there’s no point recognizing this fact if no one else on the team does, so you may have to take yourself out of that situation, and look elsewhere, but at least you’ll know what you’re looking for.

Learning To Think Functionally: Types, Maps and Sets

Download Sample Project 3.5 KB

In the most recent post in this series I implemented Tic-Tac-Toe using recursion to find the best moves. The point of that post was the recursion and I took the simplest approach I could think of to represent the actual board and moves.

I used two lists of ints, one for each player’s list of occupied squares. The board itself wasn’t explicitly represented at all, it could be inferred from the two lists.

One problem with this is that there are only 9 possible squares, but our lists could hold any integers, or could both hold the same integer. It was not only possible but quite easy to represent an invalid state.

For this post I’ll turn to this issue and try to use the type system to give us a more sensible model of the game.

    type Player = X | O

    type Position = TopLeft | TopMiddle | TopRight 
                    | MiddleLeft | Center | MiddleRight 
                    | BottomLeft | BottomMiddle| BottomRight

    type Board = Map<Position, Player Option>

I’ve represented a Player as being X or O, and the Position type has nine possible values, i.e. the nine squares on the board.

We move away from the idea in the last post of just storing each players squares and instead we use a Map from Position to a Player Option. This means that we can look at any square on the board and see if it is owned by X, O or by neither.

This fairly simple change has immediately solved the two invalid state issues I described above. A given square can not belong to both players simultaneously, and player can only own squares on the board, we can’t just “invent” new positions as we could by putting extra integers into the lists in the earlier example.

Let’s initialize an empty board so that we can start playing.

    let NewBoard: Board = 
            Map [ (TopLeft, None); (TopMiddle, None); (TopRight, None); 
                  (MiddleLeft, None); (Center, None); (MiddleRight, None); 
                  (BottomLeft, None); (BottomMiddle, None); (BottomRight, None) ]

I couldn’t figure out how to do a nice “For All” Positions, Map to None, other than by using Reflection which just seemed cumbersome for such a simple task.

Wins are defined in the same way as in the last post, but it’s a little more readable now because each position is named.

    let wins = set [ set [ TopLeft; TopMiddle; TopRight ] ;
                     set [ MiddleLeft; Center; MiddleRight ] ;
                     set [ BottomLeft; BottomMiddle; BottomRight ] ;
                     set [ TopLeft; MiddleLeft; BottomLeft ] ;
                     set [ TopMiddle; Center;  BottomMiddle ] ;
                     set [ TopRight; MiddleRight; BottomRight ] ;
                     set [ TopLeft; Center; BottomRight ] ;
                     set [ TopRight; Center; BottomLeft ] ; ]

Now we need a little helper function that will allow us to find all the squares with particular contents, i.e. All Empty squares, all squares for X or all squares for O.

    let FindPositions (player: Player Option) (board: Board) =
        board
        |> Map.filter (fun _ mark -> mark = player) 
        |> Map.toSeq
        |> Seq.map fst
        |> Set.ofSeq

This looks a little involved, but it’s very simple. We filter the board (which you’ll recall is a Map from Position to Player Option). We look for all squares with a mark that matches the Player we are looking for. This will work whether we pass in X, O or None.

The filter returns another Map, we want to split the Keys (Positions) from the Values (X, O, None). We convert the Map to a Sequence of Tuples and then use fst to extract the Key part of each Tuple. Don’t let Map and map confuse you. Seq.map is just the plain old map function that you know and love.

What we ultimately want out of this is a Set so our last step is to convert our Sequence of Positions into a Set of Positions. With all that written, we can do the following.

    let Available = FindPositions None

Because we’re using Sets instead of Lists, our IsWin function is a little simpler than in the previous example. We no longer have to write functions to decide if a list contains another list. We can simply use the Subset behavior of Sets.

    let IsWin (player: Player) (board: Board) =
        let playersSquares = 
            board |> FindPositions (Some player)
        wins 
        |> Set.exists (fun win -> win.IsSubsetOf playersSquares)

And our IsDraw function is also simple. If a given position isn’t a win then it’s easy to check if it’s a draw, simply check if there is no where left to move. Note that both the Available and IsDraw functions use “Point-Free Syntax“.

    let IsDraw = Available >> Set.isEmpty

Another helper function now, for a given player we need a way of knowing the opponent, this will allow us to toggle back and forth between players as we search recursively for the best move.

    let Opponent = function
        | X -> O
        | O -> X

Actually making a move is just a matter of adding a mapping from a Position to a Player (or Some Player as Options would have us say).

    let Move (player: Player) (position: Position) (board: Board) =
        board.Add(position, Some player)

Note that the last argument to Move is the Board, and the function also returns a board. This allows us to use the following syntax.

        let pos = 
            NewBoard
            |> Move X TopLeft
            |> Move O TopMiddle
            |> Move X Center
            |> Move O TopRight

After running the code above, pos will be a board containing X in the Top Left and Center and O in the Top Middle and Top Right. The rest of the positions will be empty.

And now, the big step, the functions that actually do the work of funding best moves for a given Player faced with a given Board.

    let rec Score (player: Player) (board: Board) =
        if (IsWin player board) then board |> Available |> Set.count |> (+) 1
        else if (IsDraw board) then 0
        else 
            let opponent = Opponent player
            let opponentsBestMove = BestMove opponent board
            let newBoard = Move opponent opponentsBestMove board
            -Score opponent newBoard

    and BestMove (player: Player) (board: Board): Position =
        Available board
        |> Set.toList
        |> List.maxBy (fun m -> Score player (board.Add(m, Some player))) 

Apart from changes to use Sets and the new Move Syntax, this code is like that in the previous post.

There is one small but significant change. If the earlier code had a choice between a definite win in 1 move or in 2 moves it didn’t care which it took, both were definite wins. This led to it passing up winning moves and winning on the next move instead. It looked a little odd.

To solve this we change how we value wins. Instead of assigning a score of 1 to all wins, which we did in the last version of the code, the Score is now based on how many empty squares remain when the game ends. This makes quicker wins more valuable.

I kind of liked the way the previous implementation sometimes seemed to toy with its victim, so I wouldn’t necessarily call this fix an improvement.

Stop focusing on Agile, fly the damn plane

Constant Learning
Being a software developer means constant learning. The technical landscape is always shifting. We have to run to stand still. We know this. We accept it. For some it’s the very thing that attracts them to the profession.

I’ve learned lots about software development in the last few years.

  • How to automate builds
  • How to automate tests
  • Object Oriented Programming/Design
  • Functional Programming/Design
  • Operating Systems
  • Programming Languages
  • Frameworks
  • Version Contol Systems

I’ve tried to embrace Agile, hell I’m even a certified Scrum Master. I attend conferences, speak at conferences, read lots and blog a little.

Despite all this I feel I am a worse “Software Developer” than I used to be. Which can be partly explained by this quote from John Archibald Wheeler.

“We live on an island surrounded by a sea of ignorance. As our island of knowledge grows, so does the shore of our ignorance.”

In other words, the more you learn, the more stupid you feel.

This, combined with the Dunning-Kruger effect suggests that feeling like we’re getting worse, even as we get better might be understandable.

But that’s not it. It would be great to explain this all away, pretend it’s all in the mind, but I don’t think it is. I believe I am actually a worse developer now than I used to be. Less productive, less focused, less comfortable.

And, I think I know why.

Fly The Plane
In an emergency pilots are trained to remember that their first priority is to “fly the plane”. It may seem odd that they need to be reminded of that fact, but it’s incredibly easy to become focused on an instrument that doesn’t work and forget to keep the plane in the air. On December 29th 1972 Eastern Air Lines Flight 401 crashed into the Florida Everglades with 101 fatalities. The flight crew were all focused on a burned out landing gear bulb and failed to notice that the autopilot wasn’t maintaining altitude.

They weren’t bad pilots, or bad people, they made a mistake, a mistake that we all make, all the time. The consequences of focusing on an immediate issue, forgetting to fly the plane, trusting that the autopilot had their back was catastrophic. They paid the ultimate price.

The consequences to the rest of us of making a similar mistake are far more benign, but there are consequences. I don’t think I’m alone in letting a focus on “building software the right way” distract from the real job of “building software”.

For me, it all started to go wrong when I started learning TDD.

Devouring everything I could read about TDD gave me a glimpse of an Agile world, of continuous integration, continuous deployment, executable specifications, distributed version control systems and feature branches. A world where the software development process worked. A magical place where you could pick requirements off the trees and working software flowed like a mighty stream.

Knowing that such a magical place existed became a curse. It made me resent the daily frustrations that have always blighted software developers. It made me feel bad every time I wrote code that didn’t have tests. It made me obsess over design and clean code to the point that sometimes I froze unable to move forward, It made me waste hours trying to automate things that absolutely needed to be automated, but not at the cost of shipping software.

Tools Tools Tools
The Agile manifesto proposes “Individuals and interactions over processes and tools”. And yet, processes and tools are deployed in ever growing numbers in an attempt to “be agile”. I’ve spent a huge chunk of my time on tools like Team City, Jenkins, Git, Subversion, Testrail, Rally, Jira, FitNesse, RSpec, NUnit, Vagrant, Puppet, VirtualBox, and all that before we even get to a programming language.

You can study all of those tools, learn about 20% of each of them and still not know a damn thing about delivering software other than it’s really hard to get tools to talk to each other.

A Call to Action
Here’s what I’ve started doing, and if my sorry tale sounds familiar you might like to join me.

Stop.

Go and build a product. Any product, but make it a complete product, it can be small but it must actually do something. I’ve started with a tool to rank the pool players in our office using Elo Ratings.

Don’t use any tools other than your IDE.

Open up Visual Studio or RubyMine or whatever and build a product.

Don’t write unit tests, don’t automate the build or the deployment. Don’t try to be agile.

When you have a fully working product, build another, and another, or built the same product again.

Forget the Red-Green-Refactor Rhythm, get into the rhythm of building working products, not functions, not programs, “Products”. Don’t just throw any old crap up and call it a product, apply a little polish. Pretend you are delivering for a client. Start by delivering the core value of the product and then improve it.

Do as much as possible manually so that you get your mind back to the bare bones of building software. What do you actually need to do?

Get faster at delivering. You should be able to build a small app in a few hours. Build the same app multiple times, Katas don’t have to be about Test Driven Development of tiny functions. Do a “Ship A Product” Kata, build a product in an hour, by hand, then throw it away and build it again.

Once you’ve got that rhythm going then, AND ONLY THEN, add in an automated build. When you’ve got that working then AND ONLY THEN add in automated tests.

For my first stab at this I didn’t even use version control, I put the code in dropbox.

Don’t do anything because “It’s the right thing to do, or it’s agile”, only do things because you can see that it makes sense, makes you faster, makes life easier, solves an ACTUAL problem.

Minimum Viable Product
Working Software is the minimum viable product from your software development process. An automated build that delivers nothing is just wasted time. You can spend a long time creating an all encompassing Walking Skeleton and never start on the product.

Focus on actually completing some small products. Figure out what you really need. Evolve your Walking Skeleton from first principles.

This isn’t an anti-agile or anti-tdd post. Quite the opposite. We need to take an agile approach to being agile. Working software is our green light, that’s the baseline. If adopting any agile practice hinders your ability to deliver working software then revert, get back to green and try again, or try something else.