Often it is useful to step back from the day-to-day work and to ask ourselves in a more global sense “how are we doing?” This is useful not only at the team level, but also in order for the organization to understand the progress of the implementation. I've found that developing simple surveys, aimed at the gathering data on our criteria for success is a useful way to:

1. Ensure we are focussing on the right things
2. Increasing transparency of the implementation
3. Finding out specific issues that we need to address

In one of the shops I worked in we developed was a series of surveys that we conducted anonymously to get the feedback we needed. We conducted these surveys periodically (every 6 months in the early days) and sent the same set of questions to anyone that was “doing Scrum”. We trended the results over time to see what kinds of changes were occurring. Theses surveys were not intended to replace retrospectives or day-to-day contact with the team. It was mostly used to step back from these more day-to-day activities.

Each question was framed in terms of “compared to what you were doing before …” and in addition to a simply quantitative response we also supplied room for comments. In general we found the quantitive data was useful for management reports while the comments allowed us to work on issues important to the team.

Running the survey required a little positioning at first (anonymous, we want information, we will do something about it) especially in the early days as people were skeptical about management intent in some cases (I asked a survey question once and found that 20% of folks believe the anonymous survey isn’t and that decisions about future employment at the organization would be influenced by the survey).

Attached is a sample “result” from one of these surveys which serves both as a starting point for the questions you might want to ask, as well as a view of the kind of data that we can generate. Some comments on this set of questions:

• We asked the teams “what kinds of things are you interested in finding out?” and used this input to drive some of the questions. This, in particular, drove the questions like “effectiveness of Sprint Planning” and “wasted / throw away work” (the organization was trying to overcome a culture of “big, upfront planning” and people worried, for example, that if they didn’t plan as much then there would be a lot of re-work).
• We also asked our executives about what they wanted to know. This lead to questions such as “how much we achieved” as there was a feeling that because of Scrum, teams would actually end up doing less. This turned out not to be the case and the survey helped work this issue.
• Some of the questions in the attachment may not make sense in your situation. For example “How do you feel about moving into a common room?” may not appropriate as perhaps you haven't done that. We also put in the question “If you received Scrum training in the last six months, how would you rate the training?” because we had gone down path doing internal training (I was doing it all) and wanted to gauge its effectiveness and determine what we needed to address.
• If I was doing this now I might want to also add in something about “Quality of the Product Backlog”, “interaction between team, SM and PO” but I suspect you will want your own questions in there as well.

I would also seriously consider a final questions like “You are king / queen for a day. What two things would you work on / change?” to make sure we are not being too directive in terms of the results.

The open ended comment questions typically required additional work and analysis but it is definitely worth the effort.

One thing with surveys is that when you run them, you set an expectation that some things will be worked. We followed up the survey with open reporting of the results and management identifying a couple of specific things they worked as a result of the survey.

See running_your_first_scrum_team_survey.xlsx