Category: process improvement

Total 10 Posts

Video Notes – Tools for Continuous Integration at Google Scale (GTAC2010)

Tools for Continuous Integration at Google Scale (GTAC2010)
Nathan York
29 min, 55 sec
http://www.youtube.com/watch?v=b52aXZ2yi08

Slide Notes:

  • Software Engineering Gap – lots of work on platforms and compilers, lots of work on apps. Middle (build systems, etc.) is often ignored (reach good enough and then move on)
  • Common Build System Issues – Incorrect, Slow, Cumbersome, Under-maintained
  • Why Build Systems Matter – Engineer Productivity, All about feedback
  • The Challenge At Google – 6000 engineers and one code base, everything built from source, development on mainline, extensive automated testing
  • Rough Developer Workflow (flow chart)
  • Better Build System Needed – optimized and tuned build languages, dependency analysis and scheduling, leverage infrastructure. Must be correct AND fast.
  • Inputs, Outputs and Actions – Content addressable storage (by digest of content), use relative paths, eliminate global state
  • Scaling Source Code Access – FUSE based file system. Most code needed for read only, on-demand syncing and caching, all source in the cloud, content digests as metadata
  • Making Builds Fast – Distributed builds in the cloud – built in arbitrary location
  • Scalable Distributed Builds – Caching key to scalable build. If inputs (from digest) and actions are same as previous, return prior result.
  • Scaling Build Outputs – FUSE based file systems, all output in the cloud, shared across builds and users
  • System View – builds appear local but are in the cloud
  • Platform for Automated Testing – Executing a test is just another build action.
  • Results – 20+ code changes per minute, 65K builds per day, 10000 CPUs, 50 TB memory, ~1PB output every 7 days, 94% cache hit rate.
  • Estimating build tool savings 2008 to 2009: Saving ~600 person years
  • Conclusion: build system is a core component of software engineering
  • Questions

 

By - On 2011-08-27 - in conferences, process improvement, software engineering, source code control

The Power of Ten – Rules for Developing Safety Critical Code

I was reading an article when I saw a reference to how NASA’s Jet Propulsion Laboratory (JPL) was using 10 simple coding guideline to develop safe code. The article referenced the following URL: http://spinroot.com/p10/. Going to this site, you will discover that the work was originally published in the June 2006 issue of IEEE Computer in The Power of Ten – Rules for Developing Safety Critical Code by Gerard J. Holzmann.

The paper and site describes 10 rules:

  1. Restrict to simple control flow constructs.
  2. Give all loops a fixed upper-bound.
  3. Do not use dynamic memory allocation after initialization.
  4. Limit functions to no more than 60 lines of text.
  5. Use minimally two assertions per function on average.
  6. Declare data objects at the smallest possible level of scope.
  7. Check the return value of non-void functions, and check the validity of function parameters.
  8. Limit the use of the preprocessor to file inclusion and simple macros.
  9. Limit the use of pointers. Use no more than two levels of dereferencing per expression.
  10. Compile with all warnings enabled, and use one or more source code analyzers.

Check the site and the paper out.

 

By - On 2011-01-29 - in process improvement, security, software engineering

Douglas Crockford – Quality

A coworker pointed me to the following video: Quality Software Development by Yahoo Architect Douglas Crockford (181 MB). The presentation from from the Yahoo 2007 FrontEnd Engineering Summit (March 7-8, 2007).

Below are my notes of the slide titles.

  • The Software Crisis
  • Craft vs Engineering
  • Computer Science has not taught us how to manage software projects
  • Software Construction (Good and Bad Analogy)
  • Nature of Software
  • Programming is Difficult
  • Lack of Metrics
  • Lines of Code – not good
  • Programmers are optimists
  • Programmers do not understand how they spend their time
  • Actual time typing is pretty small
  • Skeptical of anything that requires more keystrokes
  • Programming is a social activity
  • Cost of Innovation
  • Legacy
  • Leaps (of productivity? software capability?)
  • Object Oriented Programming (History)
  • Failed Leaps
  • Software does not have enough self awareness to be afraid of bugs
  • Bugs
  • Snake Oil / Silver Bullets
  • Mythical Man Month (1975)
  • Literate Programming (Knuth)
  • Significant difference in individual ability
  • Surgical Team (Harlan Mills)
  • Incrementalism
  • Beta (Perpetually Unfinished)
  • Application triad (skill, technology, requirements)
  • Feature Cost
  • Code Value
  • Code Quality (Micro and Macro View)
  • Simplest thing to enhance value of codebase – make more readable
  • Yahoo Javascript coding convention
  • Programs are a medium of intentional communications
  • Good architecture – necessary structure to keep from collapsing
  • How do we change a correct program into another correct program?
  • Cruft – “Software Scar Tissue”
  • Causes of Cruft
  • Bloat – “Software Cancer”
  • Insecurity
  • Cruft accumulates -> complexity groups -> progress slows
  • Refactoring
  • Sometimes it is best to start over
  • The pain of the crash
  • The illusion of completion
  • An experienced team can cross that ground again very quickly
  • Conclusion

 

By - On 2010-07-17 - in process improvement, project management, requirements, security, software engineering

Testing and December 2009 MSDN Magazine

Even if you are not working on the Microsoft platform, the December 2009 MSDN Magazine has some articles which can be of use to all testers.

In particular, Pairwise Testing with QICT by James McCaffrey provides enough insight into the QICT tool that it can be easily ported to any platform. While Automated Unit Tests for Legacy Code with Pex by Nikhil Sachdeva is specific to Microsoft technologies, Pex is a tool that all testers should become familiar. Finally, Using Agile Techniques to Pay Back Technical Debt by David Laribee has tips that are useful to most test organizations.

 

By - On 2010-01-23 - in process improvement, professional development, test tools

Jason Fried of 37Signals – Business of Software 2008

One of the things I love is being able to see presentations from conferences I was unable to attend. A coworker passed on a link to this video (55 minutes) from the Business of Software conference. In the talk, Jason Fried of 37Signals, makes of Ruby on Rails and Basecamp, speaks on many aspects of running a software company and keeping teams working effectively.

Here are some of the slide titles, topics and items Jason discussed:

  • Planning is vastly overrated – no roadmaps, specifications, projections. Get rid of distractions. Functional specification does not reflect reality and leads to an illusion of agreement.
  • Decisions are temporary – optimize for now
  • Red flag words – need, can’t, easy, everybody, nobody. These are the words that cause projects to be late
  • Interruption is the enemy of productivity – the closer the team is physically, the less you get done. Interruption is not collaboration. A fragmented day is not a productive day.
  • Underdoing – target non-consumption
  • Find the right size – imagine the software as a physical item
  • Follow the chefs – what is your cookbook?
  • Always be questioning – Why are we doing this? What problem are we solving? Is this actually useful? Are we adding value? Will this change behavior? Is there an easier way?
  • Give up on hard problems – there is an abundance of easy problems
  • You’re an editor – Curate your product. Say no to more things than you say yes.
  • Work Less
  • Q&A – Starts 28 minutes into the video

An enjoyable video, especially the first half before the Q&A session.

 

By - On 2009-09-07 - in conferences, process improvement, professional development, project management, requirements, software engineering

Hanselminutes – Test Driven Development is Design – The Last Word on TDD

In show 146 of Hanselminutes, Scott interviews agile coach Scott Bellware.

It is an interesting conversation about whether test driven development is a misnomer, the misuse of the word testability, test smells, etc. A valuable conversation for testers to consider when discussing ways to improve the quality of a product. (Especially if someone asserts that additional testing is not required since "we use test driven development".)

 

By - On 2009-03-22 - in process improvement, software engineering, system testing, web testing

Creating Opportunities to Incorporate Change

I attended a breakfast presentation by Steve McConnell (http://www.construx.com) entitled “Legacy of Agile Software Development”. A phrase that caught my attention was “creating opportunities to incorporate change”.

As we all know, The Agile Manifesto (http://agilemanifesto.org/) states valuing “responding to change over following a plan”. Steve’s presentation summarizing agile values had “responding to change” struck out and replaced with “creating opportunities to incorporate change. This struck a chord with me. I have seen many teams that call themselves agile (although I would argue that they were not…) who were resistant to responding to customer requests. Usually, they delay the request stating some rule of the methodology they are following. Almost always, the methodology would allow them to respond to the change — “create the opportunity to incorporate change” — yet the team resists. You can lead a horse to water but can’t make it drink.

 

By - On 2007-03-25 - in process improvement, software engineering

Lessons from Improv

Last week, I attended an ASQ dinner meeting where the speaker was Izzy Gesell (http://www.izzyg.com/). The title of the talk was “Tools for Transformation”. I think I would have named the talk “Lessons from Improv”. Izzy is a former improvisation artists and he applies the lessons and techniques from improvisation to the business world.

Izzy’s talk was dynamic and interactive. I took away 3 items which can be applied to a software (or any other) team:

  1. Acceptance – Accept what is given and move forward
  2. Focus – Concentrate on the essential
  3. Trust – In the process and the people

The idea behind acceptance is instead of complaining or trying to change the unchangeable, it is more productive to take what is given and move it to the next level. This does not necessarily imply agreement. Concentrating on the essential – Focus – I believe is a key attribute of all the successful teams I have been a part of. In the case where the team was not performing well, a lack of focus was definitely a component. And finally trust. Once again, the best teams operate in a an environment of trust.

 

By - On 2007-03-23 - in process improvement, project management