Building the community spirit
What would you say is the right answer? The most important thing that drives progress when building products is…
Testing once you’ve built something
Silence while you’re building
Communication at all times
If you’re a product manager at a large museum, your core team are not only the developers, designers and content creators working out what users really need. It’s also the staff around you who have knowledge of internal processes and existing products.
I found that to build fast, you simply need to communicate at all times. It might sound obvious, but it’s pretty much impossible to build a brilliant product if you don’t work closely with stakeholders.
You can try.
But chances are, it will be a disaster.
My previous blog post discussed how you can implement Agile in a museum environment. But once you’ve done that, stakeholders want to know that you’re going to deliver – and deliver the best product.
At this point, you need to win hearts and minds, roll up your sleeves and build… trust.
So how do we do this? Here are my eight takeaways from building our recently launched events programme What’s On that you can try.
1. Showcase, AKA when you share the work with stakeholders
Building a product culture is made up of a community that understands:
Its values (we focus on the best user experience, usability and accessibility for our users)
Structures and frameworks (product ownership, agile heartbeat, website steering group)
Approaches (user testing, sprints, spikes, retrospectives, releases)
A culture is based on trust that those values, structures and approaches are a means to building the best product. So how do you make sure that these become known at the Museum?
Every two weeks, at the end of each sprint, we make the effort to reach out to our colleagues and invite them to see what we have worked on. We call this a showcase.
Be this proud of what you’ve built in each sprint – and show it!
For us, a showcase is not a series of elaborate presentations, it’s live links on our website or our testing environment. We show them the new or improved features, explain how they function and why, what challenges we encountered building them and how we overcame them.
We found that when you share what’s behind the closed doors, people start appreciating the challenges you face – even when you’ve just spent half of the sprint bug fixing.
2. Collaboration: When you actually build with stakeholders
A severe lack of documentation around how three systems support our What’s On meant there was a very big piece of product logic missing in our understanding of events. As a product manager, I often felt like a secret agent hunting for clues.
For example, our events programme originates in Galaxy, passes through an ecommerce platform called Magento, where events acquire an ‘active’ status and tickets, and then land in Tycho, our content management system. Here, embellished with copy and images, they are readied for publishing.
I managed to get access to Magento as an admin user and figured out some gaps to help flesh out our API logic, but couldn’t get Galaxy installed on any of the machines in our department.
Not being able to access the origin of the events data in Galaxy was frustrating. When logic testing feels more like a pub quiz in a foreign country, it’s not the type of product testing you want to be doing.
I needed stakeholders from our Bookings and Membership departments – who work everyday with Galaxy – to set these events up for me. I wrote a series of scenarios, sorted these admins with a test Galaxy environment and instructed them to set up test events.
Although the testing took a long time, it was a fantastic trust-building experience – not only did the stakeholders suggest more scenarios we should test, but without their effort, knowledge and wisdom, I wouldn’t have been e able to decipher some of the results of the testing.
With their help, we wrestled the scenarios and nailed the product logic. A real victory feeling.
Get your stakeholders to champion you – when you help them solve their problems.
So, if you’re faced with multiple systems, I’d recommend getting access to them all – early, regularly and ideally, on your own machine. If you can’t, see this as an opportunity to build a meaningful relationship with the people who know them.
3. Learning: When stakeholders train you
If you don’t know how a system works, roll up your sleeves and get stuck in. Although I couldn’t get access to Galaxy, I needed to find a way to see it in action and get trained.
Through our showcase attendance, I came across the wonderful Amrita in the Apps team in IT. To my luck, she worked in Bookings before and knew Galaxy inside out. I asked if she could train me on Galaxy and she spent half a day with me, during which I set up two of the event tests from the scenarios I mentioned above. The best thing?
When she said: “I really miss setting up events, shall we set up another one?”
When you find advocates around the Museum who are willing to help you, hold onto them. They are literally the mythical unicorns you need to work with.
4. Milestones: When stakeholders convene towards a goal
There is nothing more heartbreaking that building a beautiful product, handing it over and then seeing it used the wrong way. For us, the front end logic is crucial in how we manage user expectation and experience on the site. It’s important that all stakeholders understand it.
Finding ways for stakeholders to become familiar with What’s On came in two forms – practical training and content migration.
A big challenge was around content migration. As we built Tycho (our content management system), we realised we couldn’t pull all event content across from Magento to it and expect events to look amazing on the front end without editing.
This was an undertaking that caused a bit of a hoo-hah across the Museum. Who would be responsible for editing events? How many events were there? Who would possibly have the time for it?
Looking back, this challenge was a blessing in disguise, and a true communal undertaking. It gave us the perfect opportunity to iron out processes around event ownership.
Once we articulated what content needed migrating, more than twenty-five admins stepped in across Membership, Marketing, Learning, Visitor Experience and Accessibility and became event editors. It was the perfect user acceptance testing of Tycho – while they were editing, bugs got discovered and ironed out right, front and centre.
5. Open surgery: When you swing the doors open
When admins started to use Tycho, we also started weekly ‘Open Surgery Sessions’. This was a window of time when admins could come into our office, report issues and chat to us. Not only was it a great way to build relationships, but also to understand the issues, and how and why they occur.
It’s invaluable to see users actually use your product too. Inevitably, you spot user pain points – things on the page that are confusing, take unnecessarily long to complete, or require too much effort.
These are opportunities to make your product easier and more intuitive to use. We noted them all and if these were quick wins we could slide into our workflow, we implemented them as we sprinted.
If they weren’t critical for launch, we added them to the Tycho backlog. Though ‘Backlog’ is notoriously known as ‘Months After What’s On Has Launched’, we have found a way to slice these up, and add some of them to our current sprints and other product work.
6. Training: When your stakeholders become new trainers
How do you train 25 event admins on using Tycho? It was impossible to do this one on one. It also felt unrealistic to achieve this in just one session – plus I wanted to avoid a situation that felt like herding cats. In the end, we created a ‘Train the trainers session’.
Train up your stakeholders to become ‘super-trainers’ responsible for passing the knowledge on.
Made up of a structure of super-admins (a couple from each department), who then became responsible for training the rest of their team, they also ensured content was peer-checked before publishing.
Once in their hands, they figured a structure of publishing events and the entire event set up process was re-thought, decided and set in stone.
Although content migration wasn’t without its challenges, particularly around consistency, we tried our best to create writing style guidelines, design guidelines, as well as a troubleshooting guide around diagnosing issues.
With the launch of a much more visual What’s On, we also identified roles missing around the product, which we continue to discus at an organisation-wide level.
7. Reveal: When you find low-stress ways to launch
When you know that bringing down the old What’s On can have serious financial implications for the Museum, you really start to think about launching safely. It was the brilliant mind of one of our developers who thought of a plan of launching Tycho months ahead of the public being able to land on the new What’s On. The new What’s On was effectively run in parallel with the old one, but kept secret behind a cloudfront wall.
We called it vam.ac.uk/whatson2, which was only accessible from within the Museum building and required a log on. This way, during the content migration process, admins could log into the admin view of Tycho, publish their events and see how they were going to look on the new What’s On.
Slowly, we realised our launch plan revolved around having to fill it out event content, iron out bugs and issues to do with ticketing, and when ready… not launch.
Instead, we would change the url to vam.ac.uk/whatson, take off the veil of secret and reveal the new What’s On. This removed the risk of shifting from a test environment to live – the site was live behind closed doors for months.
Reveal a new launch, there’s no need to do a hard switch on the day.
Our backend supplier, who was in charge with what happens behind the book now button, did the same. However, for the duration of UAT, we tested the ticket flow in their testing environment to ensure all functionality was working as expected.
At the time of launching, the live links got connected up by pointing to the right urls and the new What’s On was born.
Despite launching the day after the press night of the Pink Floyd Exhibition, and traffic being at its all time high, there was no downtime and no disruption to What’s On.
How did this launch process impact stakeholders? As soon as they understood what the reveal actually meant, they became relaxed about the launch.
8. Sweat the details: When you test with stakeholders
The most rigorous testers are your stakeholders so buckle up and use them.
Ensuring stakeholders were confident with the state of our product before launching was critical. User acceptance testing was therefore paramount with What’s On.
We organised numerous sessions for stakeholders to come and test the product and recorded their findings on post-it notes.
Though it uncovered bugs in a very productive way in the initial stages of UAT, we needed to be more systematic as we moved towards the launch date. Our tip? Find the one person in the Museum who will pedantically test every edge case and who has the complete trust of your product owner.
For us, we couldn’t have done it without a member of the CRMS team, who was – as an added bonus – also involved in launching the previous version of What’s On. Rock’n’roll.
When are you ready to press the green light?
Close to the finish line it became a meticulous process of scenarios passing or failing, constant re-prioritisation of what was acceptable to launch with, and a huge amount of patience. The pay off?
No launch down time and to this day, no significant hair-raising, revenue-endangering, safety-ruining, penalty-provoking bugs.
So to summarise, in order to build trust with stakeholders at your museum, you can:
Find ways to share your work – show them what you do and how you do it
Involve them in building – if you don’t understand something, test together
Let them train you – what you don’t know is a chance to ask for expertise
Have a common goal – you’re all in it together, what’s their bit?
Open the doors – invite them to come to you and share what needs fixing
Organise training sessions – make sure they know how to use the product
Reveal the product – explain to stakeholders how you plan to launch safely
Test with them – let them have their say in what’s critical for launch
Have you got any more techniques? It would be great to hear your ideas.
This post was originally published on the V&A Blog in September 2017.