Scrum, Kanban, Scrumban or Just Ban Scrum? (Part 2 of 2)

Part II: Some Solutions

Continued from Part 1, with the same disclaimer applied.

 

Collaboration and Distributed Scrum

If you’ve been in a successful Scrum Team with geographically distributed members, I would really like to hear from you, O Venerable Master. After all, team effectiveness has been known to diminish with as much as 2 meters of physical separation. If you must do Distributed Scrum (hopefully as a transitory step), then here are a few tips that will (I hope) save you the trouble of learning the hard way:

1. Obviously, ensure everyone on the team is fully aware of language and cultural differences, especially the Power Distance Index (if that sounds familiar, it’s from Malcolm Gladwell’s Outliers)
2. Email is only 14-17% effective as a medium of communication (Videoconferencing helps)
3. Further, information is lost exponentially as a function of distance and overhead of tools used (simply put, people are inherently lazy…)
4. …Which may prevent you from achieving fully cross-functional teams without specialists
5. Agile at scale requires trust at scale
6. Everyone needs to have access to the same infrastructure (at the same latency) and an identical development environment
7. Get Agile Evangelists (SM, Coaches, stakeholders) at every location and get them to agree on your organization’s Scrum process
8. Eliminate mixed signals, possibly by having a Proxy Product Owner
9. Empowerment and Autonomy are useless without Feedback
10. Establish an Exchange Program where [at least some] developers from different locations get to experience the reality of their counterparts (This will probably be your best investment)

The most important consideration, given all the specific limitations of your situation, must be that whatever arrangement you come up with, is sustainable. Create success stories and replicate the model. And note that Distributed Team can very well exist without Distributed Scrum.

Why Are You Doing Scrum, Anyway?

One day I’m going to be an Anti-Agile Coach. Because I’m convinced they’re so many people getting it wrong, that there’s a lot of money to be made from it. But first, some free advice: You should really consider if Agile or Scrum is really applicable to your Organization, Industry and Business Context. A good starting point is the following thought process:

You would like to adopt Scrum (or Agile) because…

1. …Your scope/specifications/customer commitments change often (If not, you might be indulging in WaterScrumFall)
2. …Your market changes often, requiring quick adaptability (e.g. Competition pops up out of nowhere overnight)
3. …You need to deliver (or release) continuously (As opposed to a one-time deployment, followed by maintenance)
4. …Your technology stack can/will change often (i.e. you haven’t invested tons of $ in license fees, and/or you will evaluate multiple options)
5. …You have the flexibility to experiment/take risks, and your organization rewards failure (or at the very least, doesn’t penalize it)
6. …Your dev team is nascent, so you expect dev processes to evolve
7. …Your industry is neither mission-critical nor heavily regulated (although there are exceptions as Nancy Van Schooenderwoert explains)
8. …You have metrics to show that Scrum would be more effective
9. …Your Admin/HR/IT teams have Daily Standups (yes, I have seen them happen)
10. …Everyone else is doing it

If you scored low on the above checklist, please consider that Iterative Software Development is still a thing, especially if you’re dealing with hardware or manufacturing. It’s not Waterfall, but it’s Scrum with a bit more commitment.

On the other hand, you might be in a field of work where you’re ready to treat software development more like Surgery than Civil Engineering (as Dan North explains in his talk).

Whatever you do, don’t indulge in Reverse Waterfall: Convincing yourselves that what you delivered is in fact what you intended to deliver.

(Please share your thoughts and experiences as a comment or connect with me on Twitter: @survivalcrziest)

 

Scrum, Kanban, Scrumban or Just Ban Scrum? (Part 1 of 2)

Part I: Some Problems

I have been playing the Scrum Master role off and on since 2008, almost all of it for geographically distributed teams. During this period, I have been lucky enough to work for 3 different companies in 4 countries. Over the years, a sinking feeling has been growing within me that we have started using “Agility” to mask unwillingness (or I dare say, often, inability) to commit. Allow me to crystallize some of the thoughts that are forming in my head.

(My usual disclaimer applies: none of the below is intended to reflect on my employer(s) or specific colleagues, whether past or present. These are my personal, generalized opinions intended for retrospection.)

Ceremonies and Metrics

First and foremost, if you don’t have Sprint Retrospectives, or in your organizational/business context, are unable to enforce accountability for, or effectively followup on, action items identified in the Retrospective, then read no further. Your Scrum is Doomed.

Sprint iteration size matters, too: “A 2-week sprint is just enough time for a mini-waterfall, and thus we are all basically whitewater rafting all the time” ( — Dan North).

If:

• you’re not collecting metrics (at the very least: Capacity, Velocity, Committed vs Completed Story Points, Release Burn Up, possibly correlation between Story Points and Hours, if you’re organization is into that sort of thing, and various forms of Lead Time)
• or not really paying attention to what they’re telling you
• and they are not visible to and understood by the whole team

then you have already failed, you just can’t see it yet.

Demo vs Release

On the other end of the spectrum, when the mandate for Scrum or Agile comes top-down, often there is excessive emphasis on ceremonies. Specifically, the end-goal of the Sprint becomes the Demo, rather than the Release. Scrum clearly dictates that no additional or specific effort must be spent on the Demo, it should be more like a natural outcome.

You can have as many Demos as you want, but what really matters for a product is Customer Feedback. Which, just to be clear, is not the same as Internal Management Feedback. Management is inherently biased in favor of your team, after all they very likely played a big part in hiring and building it. 10 internal demos are worth a single demo in front of a potentially paying customer.

The focus, then, should be on the Release of the Potentially Shippable Product Increment (complete with changelog, user documentation, installers, troubleshooting guides and quality reports) rather than an orchestrated demo in a controlled environment. This is where Continuous Integration, Continuous Deployment and more recently, DevOps comes in to the picture. If your Release Process is time consuming, unpredictable, not automated and/or has too many dependencies that are not in your control, your team is not yet Agile.

Dependencies

If your software dev team is Agile, but the rest of the organization is not, Agile is going to fail. By rest of the organization, I mean other dev teams that they have dependencies on, manufacturing, HR, Admin, IT — every single department in the organization needs to understand, live and breathe Agile. Because if they can’t respond to change effectively, then they will hold your software dev team back.

The most effective way of crippling a self-organizing Agile team is to give them full accountability, with limited or no influence over those responsible for potential impediments.

Another assured path to mayhem is when multiple teams work on a shared codebase in concurrent sprints, or when a dependency is being iterated on in the same sprint as a feature that needs to be delivered.

The Bottomless Backlog

Often in geographically distributed teams, and/or on large scale projects, the physical Scrum Task Board is replaced by a more practical virtual one. Such task boards, like those provided by JIRA Agile, Rally, Trello and several others, replace Post-Its by virtual cards, enhanced by features such as attachments, comments, time tracking and reporting. Just like any other application of technology, there is a flip side: It just a matter of a few clicks and keystrokes to add something to your Product Backlog, or to alter project plans in a significant way by moving tasks around. It’s very easy to get carried away, and often we start injecting scope creep into the Backlog, disguised as finer details of existing features (for example see User Stories vs The User’s Story).

If your organization uses different tools for Requirements Management and Project Tracking, and if there’s no strict mapping from requirements to user stories (yes, it happens 🙂 ), then it becomes very easy to lose control of the project without realizing it. A more typical scenario is working with a limited budget (usually allocated upfront),  but adding tasks endlessly — kind of like a fractal with fixed surface area, but infinite perimeter. Even with regular Backlog Grooming and prioritization, your team will still feel dissatisfied because a major chunk of the Product Backlog will end up remaining undelivered.

Finally, if you’re not in a position where you can compromise features over quality, then your Agility is severely limited.

Upfront Architecture and Specification

Probably the biggest excuse we use Agile for is to do away with high-level architecture altogether. Different Sprint teams work on different areas of the project, each optimizing locally and focused on the immediate Sprint (or two). Eventually it becomes apparent that several teams have been solving the same problem, maybe (surprise) even in different ways. The Scrum of Scrums can help, but on the rare occasions when it does happen, it is more focused on execution than design. Towards the end, more incompatibilities surface and there is a rush to “adapt the Architecture to the Business Reality”, which is basically an update of the Architectural artifacts based on the code being delivered, rather than the other way round.

So let’s recap:

1. Being Agile does not mean no design, it means less upfront design.
2. There needs to be traceability from Requirements to Specification to Architecture to Design to Execution to Validation to Deployment. It doesn’t matter what you call these things in the context of your Industry, but just because you aren’t doing Waterfall doesn’t mean you can decouple execution from specification.
3. Technical Delegation is not the same as Management Delegation. If you ask 5 teams to report progress using a specific tool, they will all report it the same way. But if you ask 5 self-organizing Scrum Teams to architect a technical solution, you will get at least 3 different results, if not 5 (or 7 with secondary recommendations).

Read on for some recommended solutions. I would love to hear your thoughts and experiences, please leave a comment or connect with me on Twitter: @survivalcrziest

Types of Managers

We’ve all seen them. We’ve worked with them. Some of us are one of them. Leave a comment if you think I missed any.

Disclaimer: Any resemblance to a single real person is purely your imagination. This is an article about Management Personas.

The Soccer Coach Manager

Angry. Disbelieving. Distrustful. Ageing. Frustrated. Abusive. Resting on decades-old laurels. Often wears a suit.

The Stayin’ Alive Manager

Dances from one meeting to another. Struggles to make it through the day. Seems to be constantly falling behind. Emails you on Saturday nights and Sunday afternoons. Is always forgetting something important.

The Stealth Manager

Never available for important meetings because he’s attending other important meetings. Never responds to emails because he has too many emails to read. Known to be sending out important emails while being away from his laptop and phone. Only seen at office parties and team building events, or anywhere else where either alcohol is served or upper management is present. Delegates everything, tracks nothing.

The Null Manager

A stealth manager who succeeded in making him/herself invisible.

The Mismanager

Destroys every project s/he touches. Announces deadlines when they are looming so close you can physically feel the heat. Generally clueless and relies on others to complete his/her sentences with facts. Then rephrases what other said as his own ideas. Following the Dilbert Principle, usually found in high places.

The Smooth Operator Manager

Wildly successful. Impeccably dressed (often in blue or pink). Never seen without sunglasses, even indoors. Highly educated, with at least one degree being an MBA. Knows his/her scotch and wine. Hasn’t travelled in anything below Business Class in a decade. Has expensive hobbies. Fitness freak. Possibly good at his/her job. Either deeply envied or deeply hated.

The Commando Manager

Ex-military, with a pair of Ray Ban Aviators. Bossy, dominating. Often mixes brown shoes with black suits (and vice versa). Well meaning but possibly anachronic. Always has a war story or joke to share. Often repeats them.

The Forced Manager

Just wants to sit in a corner with his headphones on and get some real work done. Instead spends his day juggling colorful Excel sheets, whining employees and dissatisfied customers. Despondent, maybe even dejected or depressed. Has a picture of a tropical island and/or his family in a discrete corner (or as his/her wallpaper) to remind him/herself about why they still need this job. Falls sick often because of the stress.

The Leader

The only type of manager we really need. Rose up from the ranks. Knows what s/he’s talking about. Is willing to take risks and stand up for his/her people. Good humored. Well organized. Compassionate. Possibly eccentric.

This is a Revolution, Not a Recession

“In human history, there have been three great technological revolutions and many smaller ones.  The three great ones are the agricultural revolution, the industrial revolution, and the one we are now in the middle of—the software revolution. […] It appears that the software revolution will do what technology usually does—create wealth but destroy jobs.”

– Sam Altman, “The Software Revolution”

A lot has been said and written about the great recession, the rise of the robot economy and the loss of jobs (or the creation of them in the new sharing economy – nobody is really sure yet). Let’s take a step back and get some perspective:

“Coming out the recession” and “Economic recovery”

What I think those people (and governments) who say this don’t realize is, that the jobs that were destroyed in the recession, are not going to come back. Because just like individuals, businesses too found ways of making ends meet during the hard times. This surge in demand for more cost effective labor was met by increased automation. And jobs that were automated then are not likely to come back now.

On the other hand, for decades science fiction movies and books have painted a promising picture of a utopian future where machines would do all the work and we humans would engage ourselves in higher pursuits. Yet, here we are, wanting to get our jobs back from the machines.

At the same time, we are heading towards a global workforce crisis by 2030.

Something doesn’t add up.

The Dignity of Wage

Maybe the answer is in what I call “the dignity of wage”. It’s the same reason why people are driven to crime. Our modern economy is robbing people of the opportunities to make money, while at the same time the media constantly bombards them with the notion of spending it. I believe many of the prisoners (who, by the way, in some places outnumber students) would be willing to mend their ways, if only society would give them a fair chance at earning a decent living for themselves and their families. People need something to do, to give meaning to their lives.

Especially in a society and value system in which the education and value system is so heavily employability oriented.

Maybe it’s time for a political debate on Basic Income.

“Humans will be able to move up to more ‘creative’ jobs…”

“…while machines will do the more routine ones”. The problem with creativity is, it’s not for everyone. Neither are tech jobs. Both require a certain mindset that takes years of training to master. Finally, “higher” creative jobs are only relative to “lower” mundane jobs. If all jobs became equally creative, people would have nothing to aspire to. We will be back to square one. For example, The Netherlands’s success is based on acknowledging this distinction.

Automation or Population: Pick One

There are some other longer-term issues with our current employment scenario. Life on this planet is heavily dependent on two things: oil, which we are rapidly running out of, and electricity, which, if disrupted, could instantly send us back to the stone age. Collectively as a civilization, we are not very well prepared for large-scale change.

This planet is currently on an unsustainable trajectory. And automation is only unbalancing our society more. In order to ensure continued “dignity of wage” we need to either limit automation, or limit our population. The reason is that automation is that it can transcend international borders without a visa, but human beings can not.

Sometimes this leads to strange side effects. When I was in Bangalore, the parking lot at my office had a security guard who would flag you down while entering and note down the vehicle’s registration number. Years later, someone decided it would be a good idea to install an automated parking gate, as seen around Europe and North America. I’m not sure what they were thinking, but then we had 3 security guards instead of one: one to press the button on the gate since it was installed too high up to be reachable by drivers of average Indian height in average Indian cars, one to make a manual note in the register as a back up, and a third to supervise over the first two.

Which brings me to the next point: Any transaction that involves a human being is inherently potentially flawed. No amount of automation will make our world perfect or optimal, as long as it’s still “our world”.

“Creating Jobs”

The tobacco industry employs more than a 100 million people worldwide. If we want to create more jobs, we should all smoke more cigarettes. By extension, conflict has traditionally been the biggest source of employment: In research, manufacturing, exports, destroying settlements, and providing private security while rebuilding them. As a Green Beret recently pointed out, in today’s job market a Special Forces training is better than an MBA.

So there you have it, the solution to all our economic problems is perpetual war.

Or we can take inspiration from a bunch of African kids, and grin and share it. In other words, a Resource Based Economy instead of an Employment Based Economy.

Endnote: You really should read the link about the rise of the robot economy.

Update 2015-05-25: More food for thought: Self-Driving Trucks Are Going to Hit Us Like a Human-Driven Truck

“I just have to…” Project Management

(EDIT: This post was republished by PMHut.com here)

If you’ve worked in/with a software development team, in any capacity, you have heard this optimistic statement before: “I just have to <insert_trivial_sounding_technical_task_here> and then I’m done!”

• “The new approach worked. I tested it. I just have to move this method to the other class, and then I’m done!”
• “I’ve committed my changes in a branch. I just have to re-run the tests once more, and then I’m done!”
• “I’ve finished everything. I’m ready to start the next task. I just have to update the README and then I’m done!”

Well, you know what they say: in any software project, 90% of the work is completed in the first half of the schedule. To complete the rest 10%, it requires the rest half of the schedule, plus another 50% schedule overrun, plus x extra resources, everyone working overtime and oh yeah, we’ll ship the documentation later because we were focussing on delivering the functionality… you get the point.

You know Brook’s Law? It applies to machines as well. When you write tools and post-build scripts to automate development tasks, when you adopt the latest DVCS, when you start using the latest test automation framework… you are adding resources to your project. And it has been clearly established that adding resources to a project that is already behind schedule (and which project isn’t) is only going to make it later.

Here are a few things I’ve learnt over the years:

1. Nothing on planet Earth is a zero-time activity. Your plan: “I just have to move this method to the other class, and then I’m done!”. What happens in real life:

1. • You need to get your code peer reviewed.
   • But there is nobody available at the moment to review.
   • You finally interrupt someone, because y’know, it’s just one method, you just have to quickly review it… (of course you would conveniently forget how you would feel if you were the one being interrupted)
   • What? It’s a public API? Then you have to update the acceptance tests.
   • And the unit tests.
   • Go back for peer review and sign off. After lunch.
   • OK, all done, just commit and… WTF now I need to merge?!
   • And rebuild.
   • And re-run the tests.
   • And re-start the PC because of that pesky Visual Studio bug.
   • WTF? Windows updates? Now?!
2. Murphy’s Law is as fundamental to computer science as boolean algebra. It’s a pity they don’t teach it in school.
3. Remember algorithmic analysis? When you estimate something, don’t always consider the best case. The worst case happens more often than you’d think.
4. ABC: Always Be Collecting. Data. Statistics. Your past project data will show unrealistic your estimates really are. Nothing else can convince you as much.
5. If you haven’t been able to solve a problem for too long (confused customer, unreliable dev environment, too many distractions or interruptions), then maybe it’s not a problem, it’s a fact. So adjust your future estimates accordingly.
6. Say “out of scope” more often. Just because it’s software and easy to modify, doesn’t mean it should be modified. Agile doesn’t mean the freedom to constantly change requirements, it means adapting to changing requirements (amongst other things). Changing requirements doesn’t mean freedom to change fundamental constraints. Think of it this way: if you were building a bridge, it would be acceptable to try out different road surfaces. But would you change the bridge into a tunnel because it offers less wind resistance? And should there be a need in the future, you can just drop the already built tunnel underwater? Some software change requests and developer’s grand visions are equally unrealistic, and unnecessary.
7. Life is a fractal. If you’re late every day, you will be late every month, every year… and you will feel the same way about your whole life. It happens at home too: “I just have to get dressed, and I’m ready to leave”. Real life: Get dressed, grab snack, drink water, close window, remove phone from charging, respond to missed call, tie shoelaces, oops mismatched socks, find keys, lock door, wait for stuck elevator…
8. Work-life balance doesn’t mean work-life separation. Bring your good habits to work. Inculcate learnings from your work into your personal life. Why limit Continuous Improvement only 8 hours in a day?
9. Learn from other’s mistakes. Every time someone is being a moron, they are secretly trying to teach you something. The daily standup is not only about you, it’s also an opportunity to improve based on others’ experience.
10. Most importantly: Stop saying “I just have to…”  🙂

Takeaways from Craft Conference 2014, Budapest – Day 3

Continued from Day 2, here are the talks from Day 3 in order of my personal preference (and relevance) which may differ from yours:

Complex Projects aren’t planable but controllable

by Jutta Eckstein (Slides | Video)

Sadly, Jutta’s slides aren’t available online but the talk was packed with solid advice for Project Managers. Some of them were:

• Our predictions are usually based on a coherent story, not a complex, dynamically changing reality
• Targets should be ambitions, not absolute
• Focus on the value gained, rather estimates of the effort required
• Having an  annual budget is like having a bank that opens for business only once a year
• Annual budgets are not optimal because they are never underutilized: the excess (if any) is still used up, never returned back
• Consider [event-based] rolling budgets, rolling plans and rolling control
• Check value and progress regularly
• Don’t just trust the experts, seek feedback from diverse groups
• Have different planning strategies for Roadmap (themes only), Release (features based on value & velocity) and Iteration (Stories based on value (+estimate) & velocity)

Recommended reading:

• Implementing Beyond Budgeting: Unlocking the Performance Potential, by Bjarte Bogsnes

Architecture War Stories

by Stefan Tilkov (Slides | Video)

Probably the most amusing talk of the event. Stefan shared some hilarious real-life examples of architectural disasters… some of them still very much in use (of course no names were revealed). His advice was to go back to the basics:

• Make data free of code dependencies
• If it makes you want to pull your eyes out, maybe you shouldn’t be doing it
• Better ask for forgiveness than permission
• Development environment is not the same as production environment
• Feedback, reflection, iteration

Responsibly maximizing craftsmanship in software engineering

by Theo Schlossnagle (Slides | Video)

Theo revisited the basic problems plaguing software development today and went on to share some of his experiences at Circonus. One of the tips was to treat all software you write as open-source within the organization, even if you don’t plan to release the source outside. Other highlights:

• Turtles all the way down: Software at scale is tied together with loose string & hope
• We use human language to describe software specifications, and this can be interpreted by different people in different ways (like poetry)
• Technical debt is non-linear; large monolithic components are more likely to fail because they are hard to maintain
• Reusability is good, but use the right tools for the right purpose. Accept that that right tool may not yet exist and may need to be written
• Diversity is an emergent property of scale
• Thou shalt be judged by your API

For software developers, the message was loud and clear:

Data-Driven Software Engineering

by Jevgeni Kabanov (Slides | Video)

Jevgeni went thorough the effort of analyzing 1000+ projects on Productivity, Quality and Predictability. Sadly the slides aren’t available, but the raw data they were based on is. You can still watch the video and grasp the most important points, such as the fact that the best software projects are delivered 80% of the time on schedule and still have 25% critical issues. Or that automated tests improve quality by 26%. There were also secondary interpretations such as code reviews improve architecture, not just code quality.

Jevgeni emphasized on the importance of measure & experiment over simply “doing agile”. Some suggestions for measurement were:

• Deadline misses
• Scope changes
• Blockers after release
• User satisfaction

The presentation became a bit controversial later but as Jevgeni said: “People, chill – I gathered data and presented my analysis – feel free to take the data and do a better job”. And I think this talk had the most creative closing slide ever 😉

Without Present or Past: How to Think Distributed – The Hard Way

by Endre Sandor Varga (Slides | Video)

By far the most profound and philosophical talks at Craft. If you have an interest in Distributed Systems, AI or Information Architecture, I would strongly recommend watching the video. The core concept was based on applying epsitemic thinking to systems that are distributed, concurrent, able to fail independently, and communicate over a lossy medium with non-deterministic communication delay. Endre touched upon:

• Two Armies Problem
• The final important message
• Difference between actual state and observed state
• The Omnipotent Observer (doesn’t exist, because state is always queried)
• Global introspection and self-awareness
• Cone of the past
• The present is volatile and subjective
• Going from someone knows -> someone knows everyone knows -> everyone knows everyone knows

Endre ended with this advice:

• Have a properly defined failure model
• Never assume reliable communication
• Never assume common knowledge

Functional Reactive Programming in Elm and JS

by Evan Czaplicki (Slides | Video)

Evan is the designer of the Elm language and his enthusiasm is infectious. Over the course of his talk, he built and demonstrated a simplified (yet slick) Mario game complete with physics and reaction to keyboard inputs. Elm is a Functional Reactive Programming language for web browser based GUIs and this game was a fine demonstration of concepts that Bodil Stokke and Jonas Boner touched upon in their respective talks.

There is no doubt that Elm is a game-changer. Here is another example of traffic simulation.

Further reading:

• Go try Elm!

Find the Right Abstraction Level for Your Tests

by Gerard Meszaros (Slides | Video)

If you’ve ever been haunted by the question of what level of testing is enough (and who hasn’t?) then this presentation was for you. The key message was to think in terms of not what you can add, but what you can leave out of the test: “if it isn’t essential to conveying the essence of the behavior, it is essential to not include it”. Over 39 slides, Gerard illustrated this step-by-step with one, continuous, easy-to-follow example.

Software Psychology: The Art of Listening to Code

by Bjorn Freeman-Benson (Slides | Video)

Bjorn talked about the concept of code screams: behavioral indications of a deeper problem in the system. For me the key takeaway was that you should continuously monitor your processes as well as your systems in production (e.g. by gathering usage statistics) and fix the root cause when a problem is found.

Others

Here are some talks that I missed, but which received a lot of positive feedback. Thanks to ustream.tv they are available online. Also check out #CraftConf on Eventifier.

Jackstones: the journey to mastery

by Dan North (Slides | Video)

McDonalds, Six Sigma, and Offshore Outsourcing: Unexpected Sources of Insight

by Chad Fowler (Slides | Video)

Testing the Hard Stuff and Staying Sane

by John Hughes (Slides | Video)

The Better Parts

by Douglas Crockford (Slides | Video)

Functional Examples from Category Theory

by Alissa Pajer (Slides | Video)

 

Takeaways from Craft Conference 2014, Budapest – Day 2

I had the privilege of attending the speaker sessions of Craft Conference last week, the central theme of which was software craftsmanship. There were many inspiring talks and so was the venue. Think of it as TED for software developers. The icing on the cake was free beer, complimentary Palinka, unlimited coffee and a blues band. 20% of the speakers were women (but only 4% of the attendees) and ~350 of the 900+ attendees were from abroad. The event was virtually flawless. The usual systems that breakdown at scale: WiFi, food and toilets, all just worked. Plus there were small thoughtful touches such as English translations of useful Hungarian phrases on the attendee badge. Everything about the conference was impressive, including the raffle prizes which included a R/C drone!

Other interesting features were the use of Sli.do to manage audience questions in real-time, and a live-projected twitter feed. I had the opportunity to interact with practicing or aspiring software crafts[wo]men from Ukraine, Japan (ok, technically SF), Netherlands, Sweden, North America, UK and… India!

I’ve tried to distil out the summary of talks that interested me. They are ordered by my preference, not the order in which talks were actually conducted. These are my interpretations and my views, so please bear in mind that they could be wrong or biased. There were 3 tracks (parallel talks) over 2 days, so essentially I attended only about 1/3rd of the total. Day 2 was the first day of the speaker talks… I didn’t attend day 1, which was workshops.

I strongly recommend checking out the agenda and viewing the talks that interest you online, you might find some that I didn’t attend but are of direct interest to you. To me the top 3 recurring themes of the conference seemed to be:

1. State management in complex and distributed systems
2. Better automated testing & TDD
3. The abuse of agile (in development and in project management)

How I Learned to Stop Worrying and Love Flexible Scope

by Gojko Adzic (Slides | Video)

Probably one of the most-loved and honest talks at Craft. The storytelling was simply mind-blowing (an example of agile from 1628 AD, Ducati’s experience with the Second System Syndrome, threats to competitive advantage from Google & the Russians) and so was the message. I would highly recommend watching the video, Gojko is an outstanding speaker. The highlights were:

• The most common software development methodology these days is WaterScrumFall, in which all the essential planning is done upfront by management and the development is done in a [predictable] number of sprints
• This is because the concept of agile is not attractive to management, unless they truly believe in keeping scope flexible. Therefore agile generally remains underutilized.
• Agile is not just about continuous delivery, but also continuously reacting to local, temporal and human factors
• Project plans and roadmaps should not be linear, but literally a “map of roads”: multiple options with selection criteria (like a GPS). A roadmap with a pre-decided outcome is not a roadmap, but a road in a tunnel.
• Try new things, at a survivable scale and select the ones that work (Throwing away bad code is a way to reduce your technical debt)
• Don’t just ship software, make an impact
• On the topic of outsourcing: Usually the objective is to minimize costs, so the focus is not on excellence or flexible scope
• On User Stories:
  • Are a way to delegate details
  • Avoid translating the product roadmap into a set of JIRA stories, instead consider hierarchical backlogs
  • Add a victory condition to your user stories, which is related to changing user behavior rather than complying with the existing behavior (e.g. Monitor inventory *faster*)
  • A good user story is a survivable experiment

Recommended reading:

• Adapt: Why Success Always Starts With Failure, by Tim Harford (specifically the Palchinsky Principles)
• What Customers Really Want, by Scott McKain
• 50 Quick Ideas to Improve Your User Stories, by Gojko Adzic

Agility and the essence of software architecture

by Simon Brown (Slides | Video)

This was also one of my favorite talks. I like to think of it as Minimum Viable Architecture for developers. Simon is an inspiring speaker, more so because he eats his own dog food. The premise was simple:

• Agile delivery does not imply agile architecture
• Agile development still requires upfront thinking to define the overall architecture
• The team must have a shared, consistent vision of the significant structural elements of the product
  • With this pragmatic tradeoffs can be achieved: e.g. a monolithic deployment container, containing microservices
• UML was supposed to solve this problem, but it is poorly understood, not widely adopted and has its own overheads

The solution: NoUML! Abstraction is more important than notations. A team can define their own legend for these abstractions. Design diagrams are supposed to reduce complexity (through abstraction), not increase it. The 3 things that the team needs to have a common understanding of are:

• Structure
• Risks
• Vision

In real life, we rarely have 1:1 mapping between design diagrams and code. In general, a hierarchical C4 architecture diagram can solve this problem:

• Context
• Container
• Components
• And optionally, Classes

And yes, while good architecture is a shared responsibility, it is important that only one person (or a small group of people) are responsible for maintaining the overall architecture definition.

Recommended reading:

• Just Enough Software Architecture, by George Fairbanks
• Software Architecture for Developers, by Simon Brown
• CodingtheArchitecture.com
• Risk-storming

Getting Things Done at Scale

by Amber Case (Slides | Video)

This was a talk I could relate to a lot, because of GTD and the differences in corporate cultures of large, old organizations vs smaller, newer ones. Amber is a TED speaker, and it clearly shows. She shared her experiences during (and after) the acquisition of Geoloqi by ESRI.

Only 1 of 50 people she spoke to for advice had a happy acquisition experience, and the main reasons were:

Crippling management/overhead to get simple things done	Culture clash
Founder flight	Jealousy/blocking from parent company employees
Lack of detailed transition plan	Sprinters vs Marathon Runners
Loss of passion for original product	Loss of respect and cross-compromise

The solutions discussed were:

• Pre-negotiate, don’t be vague. Predetermine your outcome.
• Learn the local language (technical terms, tools, company culture)
• Win friends to influence people. Develop trust.
• “Beta test” people via small projects. Best code wins.
• Scale teams down from unmanageable numbers to 5-6 “doers”
• Communicate. Respect. Give first.
• Small revolutions
• Distribute stress

An interesting concept was the creative use of IRC bots, e.g. to send out a daily email summary of accomplishments (!done). Towards the end, she also shared an effective way of “organic hiring”: turning contractors into full-time employees.

It’s never too late to fight your legacy!

by Mate Nadansi (Slides | Video)

Mate delivered a very strong and reassuring message: Legacy code isn’t bad, just old or over-iterated… and, with some sensible planning, foresight, and many iterations of hard work, it can be replaced by more modern code. He explained how they achieved exactly that at ustream.tv. The presentation requires some familiarity with web programming but it would still benefit anyone working with legacy code… because maintaining legacy code builds character. 😎

Programming, Only Better

by Bodil Stokke (Slides | Video)

The core message was about how the introduction of multiple [mutable] states makes programs unmanageably complex. Topics included:

• Referential Transparency
• Representation of state using numbers instead of objects to it remains immutable
• “Encapsulated state” is still state
• Effect on concurrency
• Additional complexity added by control structures in contemporary program

The presentation itself was a work of art and for me the highlight was that Bodil was editing & running code from within the slides!

Recommended reading:

• Out of the Tar Pit, a 2006 paper about Functional Relational Programming by Ben Moseley and Peter Marks

Going Reactive: Event-Driven, Scalable, Resilient & Responsive Systems

by Jonas Boner (Slides | Video)

Jonas gave an inspiring and pragmatic talk about how the nature of, and expectations from, applications have changed dramatically over the years. The highlight was that not only did he distill them into the 4 attributes mentioned in the topic of the talk, but also shared valuable insights into how to practically achieve them from a technical standpoint. Some of the approaches discussed were:

Loose coupling Never blocking Asynchronous operations Actors and Agents Futures in Scala Minimizing contention

Maximizing locality of reference Single Writer Principle The relation between scaling up and scaling out Decoupling error handling from business logic Bulkheading to prevent cascading failures Maintaining consistent latency across blue skies conditions, high traffic and failure

Recommended reading:

• ReactiveStreams.org
• The Reactive Manifesto
• TypeSafe.com
• The Network is Reliable, by Aphyr

What Makes a Good Development Process?

by Bruce Eckel (Slides | Video)

Bruce Eckel is an industry veteran an author who needs no introduction (I read his textbooks in high school). I highly recommend going over his slides, they are very comprehensive, thoughtfully put together and highly informative. He has already distilled years of experience into a few pages, and I don’t think I can add any more value except quoting the one thing that stood out for me: The cognitive load of carrying tensions prevents us from doing creative tasks well.

Acknowledging CAP at the Root – in the Domain Model

by Eric Evans (Slides | Video)

This talk dealt with a very specific type of problem (CAP = Consistency, Availability & Partition tolerance), and solving it using a domain-driven programming model. One of the interesting concepts was that of defining aggregates within a distributed system. Instead of trying to keep the entire system in a consistent state, the contents of each aggregate are guaranteed to be consistent (even if equally stale). There is only a single point of reference into an aggregate. Aggregates are contained within a bounded context and transactional consistency must not cross these boundaries. Domain events interpreted within this context cause state changes. Eventually overall system consistency can be achieved through synchronization according to a reasonable SLA.

Don’t forget to check out Day 3!

FilesDesk

2012

This started out as a side project between a couple of us at work to sharpen our C#/XML skills and generally do something challenging. A C#.NET Application + Service that enables browsing files on disk by Tag (as opposed to Path). Typical usage:

• A Tag Cloud for your hard disk
• Add/remove tag(s) to selected file(s)
• List & Filter files by tag(s)
• Monitor specified folders (ex: My Documents) for new/deleted/renamed files
• Portable XML Database

Useful for Researchers, Writers and Obsessive-Compulsive File Hoarders.

2011 Summarized – In Books

(Yes, I know it’s kinda late to be summarizing 2011 :-), but I still believe that if you have nothing important, intelligent or enriching to say, then it’s better not to say it. In other words, don’t blog just for the sake of blogging…)

Often what keeps me going through a tough day is looking forward to curling up in bed with a book at night. It is a fact (at least valid for the current state of human evolution) that we retain more of what we read in real, paper books, compared to e-books on a digital screen. Something to do with the tactile senses.

I read several interesting, thought provoking and highly influential books since last year (besides the usual classic Science Fiction). Some of the more noteworthy ones were:

• “59 Seconds: Think a Little, Change a Lot” by Prof. Richard Wiseman — I came across this in a reference at Coding Horror, and although I’m not a fan of non-fiction (and certainly not the How To sub-genre of non-fiction), I just picked up on a whim, and boy, was I hooked. I have been evangelizing it ever since, and more than a year later, I’m still discovering ways in which reading this book helped me break my [evil] habits and turned me into a better person.
• “IGNORE EVERYBODY and 39 Other Keys to Creativity” by Hugh MacLeod — A friend recommended this to me (in fact, so strongly that she gave me her copy to read), and although sometimes a little opinionated, I found the advice very relevant to the times we live in, especially The Sex & Cash Theory.
• I very quickly read “It’s not How Good You Are, It’s How Good You Want to Be.” by Paul Arden. Although the book didn’t really resonate with me (maybe because I was still reeling from the in-your-face effects of IGNORE EVERYBODY), I found some of the ideas very insightful. For example, Paul made the case that out of five sales pitches in a week, the client is most likely to pick the one presented on Tuesday, because Monday was “too early, nothing to judge by”, Wednesday & Thursday were “like eating too much chocolate” and Friday was like “feeling sick”.
• Meanwhile, Steve Jobs sadly passed away, and the time seemed to be right to read “The Steve Jobs Way: iLeadership for a New Generation” by Jay Elliot & William L. Simon. I found the book to be very balanced, using storytelling to draw focus on best practices and insights that can be applied in our own organizations. Highly recommended if you think you or your place of work could do with a fresh dose of [now legendary] iNspiration.
• I then started reading “The Go-Giver: A Little Story About a Powerful Business Idea” by Bob Burg & John David Mann, which was the first in a reading list recommended by Venkat during his inspiring talk at BarCampBangalore11 (#bcb11). This is a short book that extols the virtues of giving as the secret to success. If you’re in the business of making profit (who isn’t), this book will surely make you think.
• There was some promising talk on Facebook about “Plunnge: Reinvention for the New Generation” by Rakesh Godhwani, so I got hold of a copy shortly after it was published. I’ve read many books, and no matter how badly they are written, no matter what subject they’ve been written on, I always find something, no matter how small, to learn from them. So over the years, I’ve never regretted reading a book (except maybe my school textbooks :-p ), but after a bit of a mental struggle, I was forced to admit Plunnge was the first one. Although the subject (essentially a collection of true stories about Indians giving up successful careers to pursue their true passions) is commendable, Mr. Godhwani’s attempt at writing is best described as “otherworldy”, and clouded the true potential of the book.  The writing is heavily biased & prejudiced, almost every single page has either a grammatical or semantic error (although I must concede that some of them are popular usage in corporate India) and there was just too much credit being given to individuals who were correcting their own bad initial career choices (most of all Mr. Godhwani himself – If there was one purpose this book served, it was that of being his personal catharsis). Apparently Peak Publish, headquartered in Derbyshire, U.K. is quite accommodating with the authors they pick. Initially I was so appalled I wanted to write to both publisher & author, but as I read on I realized that (a) the publishers published it, so obviously they didn’t find anything wrong with it, and (b) the author is so pleased with himself and the topic that obviously the finer details were not important to him. So what’s the point? Don’t let your filters fail, and skip this one.
• To recover from “the Plunnge”, I went on a Malcolm Gladwell reading spree (who I think is a magnificent author), and read “Blink” and “Outliers” (which were somewhat related to the subject matter in “Freakonomics”) and that was time happily well spent. No matter where your interests or disinterests lie, if you’re human, you should read Gladwell.
• A lot was happening around that time, and I thought it was a good time to revisit the old classic “Who Moved My Cheese?” by Dr. Spencer Johnson, probably the best piece of work written on dealing with Change.
• Finally, I read the short story “Whispering Wind” by Frederick Forsyth (in his book of short stories called “The Veteran”) which I think is the perfect story. Forsyth fans and those discovering him now will equally be amazed at the level of excellence the Master Storyteller achieved with this story, especially considering that this isn’t one of the topics (or time periods) he usually writes about.
• Recently I also finished reading “Bozo and the Storyteller” by Tom Glaister, which presents a unique, thought-provoking (and sometimes depressing) view of the human condition.

I got myself  a LightWedge to and will continue reading happily into the night…

The Art of Listing

As I had posted here in 2007, I have been trying to make an effort towards paperless organization of my lists, most of which are ToDo items. The Palm device that I was originally attempting to use for this effort turned out to be a headache because of limitations on formats, storage capacity, speed, interoperability and expandability. I ended up giving it away to my cousin brother who is a student, to make his first attempt at getting organized 🙂 In the meantime, I picked up a Sony Ericsson P990i, which let me do a lot more, faster and more efficiently (Of course, that device is also fast approaching its event horizon). I found that I have so much going on in my head that often it was a pain to take out the phone, flip it open, navigate to “Tasks” or “Notes” and start typing. Going 100% paperless wasn’t working out too well, sometimes during this physical process I would lose track of my mental process (i.e. forget the idea or task that I wanted to note down). Over the years, I have arrived at the following hybrid approach, which helps me get things done effectively:

1. On my device, I maintain the following lists, in the following order, each of them almost like a Product Backlog:

1) This Week	5) Online – Stuff to do the next time I’m in front of a computer, like e-mailing somebody
2) Weekend	6) Projects – Not just software, even real-world projects like scale models
3) Next Week	7) This Quarter
4) This Month	8) This Year

Plus, the following “dynamic” lists:
a) Groceries – Since the stuff I buy every week/month is almost always the same, I just have a master list in which I keep moving things between “Pending” (unchecked) and “Completed” (checked) depending on what I run out of
b) Shopping – Other things to buy next time I’m out
c) Travel – Places to travel to on the weekends
d) Focus – 1 to 5 items I’m currently focusing on (e.g. “Get to work on time” 🙂 ), to keep reminding myself regularly

2. My phone lets me prioritize tasks within each list, on a scale of 1 – 3. Also, for example, within “This Month”, if “Pay Rent” has been completed, it gets checked into “Completed” and doesn’t get deleted. At the beginning on next month, I simply uncheck everything back into “Pending”.

3. I maintain a single sheet of pocket notebook-sized paper (more if I’m actively noting down ideas/tasks for an ongoing activity/project), akin to a Sprint Backlog, with the following:

Front Side	Back Side
Today – Things to do today (mostly at work)	This Week – Including weekend commitments
Calls – Phone calls to make + e-mails to send	Home – Things to do when I get back from work

4. Every weekend I move stuff from the “Product Backlog” (long-term list of stuff on the phone) to the “Sprint Backlog” (short-term list of stuff on paper), and *wait for it* stuff gets done! I never use more than one sheet of pocket notebook-sized paper in a week, and this way I also always have paper handy to quickly note down stuff (on the margins). Finally, in case I ever lose/damage my phone (which is backed up every 2 weeks), I don’t lose the things I had planned for the week.

Am I going overboard? (After all, it’s just a glorified ToDo list.) I don’t think so. I find that by keeping things prioritized and focused this way:

1.  I manage to get a lot more done without worrying about what I’m forgetting to do.

2. I don’t lose track of things that I would eventually like to do, but don’t have the time for right now (or this week, or this month, …)

3. Moving the prioritization and organization out of my head helps me think clearer and focus 100% on the task at hand.

But it doesn’t end there. Over a period of time (and with a lot of self-imposed discipline, I must add), I have managed to harmonize the short-term (a.k.a. “sort it when you see it”) organization of things that I come across everyday. I do this by managing the following “tags” (often as Folders, in some cases even physical file folders) across my Inboxes, Browser Bookmarks, Hard Disks and scattered notes (including those on my phone):

• BlogThis
• ReadThis
• WatchThis
• DownloadThis
• FollowUp

I visit these as and when I have the time and keep emptying them out. With the addition of lists (as notes) for Movies to watch, Music to get, Books to read and Scale Models to buy, my little universe of lists is complete!

Stuff that I learned along the way, though:

1. Hybrid is more practical than paperless.

2. We need a device (implant?) that can make a note when the wearer thinks of it (and where to put it). The interface & actions between thought and task noted are, well, so ’90s! (Note: Speech Recognition is also so ’90s)

3. It’s best to stick to simple formats like Text and CSV instead of proprietary ones (like Excel). Simpler formats are easily portable and retrievable in case of failure, and suffice for making lists. If your list seems to require a complicated format, well, simplify your list!

4. It may be a good idea to reuse Visiting Cards and such, but your handwriting needs to be tiny.

5. Evernote can probably help.

UPDATE: [2011-07-16] I have since migrated all my lists to my new BlackBerry Curve 9300.

UPDATE: [2012-03-09] I discovered Todoist, which is quite simply the Tao of using Computer Science to solve problems. Although using it means that my todo list is now in the cloud, something that I’m [still] not very comfortable with, I find it indispensable to manage long-term projects. I initially found it attractive due to its Outlook integration, which meant that I didn’t have to grapple with numerous tasks disguised as emails, but the app is constantly being improved with new features, like @labels that enable a task to be present in multiple lists. HTML5 support means my list is now available offline, and it syncs effortlessly across devices, including my BlackBerry. I occasionally take local backups with Todoist Backup.

UPDATE: [2012-09-15] With my mind emptied of the long term stuff now safe on Todoist, I have started relying more on my memory for day-to-day things. I’m also trying to do less and focus more on the important things (not to mention years of long hours have significantly shortened my “backlogs”), and try not to take on more than I can comfortably remember over the span of a few days at a time.

UPDATE: [2012-10-05] I have a new revived obsession with Whiteboards at home. I’m trying to keep it under check to avoid looking like too much of a mad scientist…

UPDATE: [2013-07-12] Since Todoist Backup no longer works and I don’t have the time to figure out what changed, I have upgraded to Todoist Premium to be able to make use of their backup service (amongst other cool features). I now exclusively use Todoist coupled with a one-page note on my phone, which has tasks for the day to week range, plus trivial items (< 2 min) that don’t need to go into Todoist. Also, the other day I found the phone I used to use before my Palm device: Nokia 6820.