External Dependencies and Risk – The Wando, SC Port Debacle
External Dependencies and Risk
Earlier this week, a seaport in South Carolina introduced a new, software-controlled entry and exit (access) system for trucks. The system is intended to streamline and speed up the controlled and secure process of moving trucks into the port and the associated terminals (the port is the entire complex while the terminals are the facilities at which ships moor to unload and load cargoes). It would be like automating the security lines at airports. That system introduction – and its unintended consequences, was reported in a story on News 2 – the local NBC affiliate. In a nutshell, rather than improving the access process and getting trucks into and out of the port quickly, the new system worked poorly and caused hours-long back-ups. Some truckers never got into the port on the first day, and the problems have extended into a second day. The system failure:
- Was the fault of the software developer who provided (apparently) bad code
- Was the fault of the port for not properly managing the software developer and insisting on comprehensive (including real-world) testing
- Cascaded from the vendor to the port to the shipping and trucking companies
- The port was the victim of a first-tier, supply-chain failure
- The shipping and trucking companies were victims of that same failure, but now at the second tier of their supply chains
- Resulted in a what could be a long-term crisis for the port – one that impacted their operations, their brand and reputation and possibly their finances/profits
- Resulted in more near-term crises for the shipping and trucking companies
The software developer and the port authority (the folks who manage the port and provide services to all the individual terminal owners and the terminal owners’ customers (shipping and trucking companies) should have planned for a less-than-perfect introduction of the new system. That plan, for problems with the introduction, should have included trigger points (conditions for which a planned action would be initiated) and specific actions to be taken by designated personnel. That plan should have included ways to measure system performance in real time so that problems could have been addressed quickly.
Back to the new system. Such a system is a great idea and one that should have been of enormous benefit to all the stakeholders, including the trucking companies and individual truckers. The News 2 story clearly reported that it was not of benefit – at least not for the first two (now, apparently three) days.
The world is becoming increasingly connected. Ships make “reservations” for a mooring spot at a terminal. That spot is typically open for one hour. If the ship misses it, the ship goes to “the end of the line” and waits for an open spot; a costly situation. Shipping is a case where time really is money. Truckers know what cargoes they are supposed to deliver to which ship and what cargoes they are supposed to pick up from which ship. They know the times that such cargoes must be delivered or will be available. Streamlining such a process to ensure that trucks and ships meet up quickly and appropriately and can save hundreds of dollars on even a single cargo container.
Unfortunately, the “parts” that comprise that entire system (ships, port, terminals, cranes, trucks, security) are not part of any monolithic, synchronized organization – they are controlled by different private and public organizations, often dozens and sometimes hundreds of them. Any system comprised of hundreds of parts that operate independently of one another and adapt and respond in order to mesh at specific times and locations is complex. That complexity must be accounted for in the development and introduction of a new system – particularly one that controls the flow of work through the system – which the system in South Carolina is supposed to do.
The software to accomplish the streamlining in the port is sophisticated and is most likely built up of thousands of modules and probably more than a million lines of code (almost certainly more than a million lines since the application I’m typing this with has more lines of code than that). Comprehensively testing such a sophisticated software program is very difficult. Truly comprehensive testing should include some level of testing with actual systems (e.g. trucks, access points, etc.).
The truckers in the long lines waiting to gain entry to the port were victims of an external dependency risk. They depended on the port to get them timely access to the port, the terminals and the ships. The ships waiting to load cargo were also victims of the same dependency on the external-to-them port access system and process. The new system failed to streamline access to the port. In fact the introduction of the system did the exact opposite – it seriously delayed all access, which created operational problems for the truckers and a crisis for the port.
In the increasingly connected world, external dependency risks are becoming more and more important. Trucking companies, as an example, don’t need to forecast the cause of an external dependency problem – they merely need to understand and develop a response for the result. Delayed access to the port could have had several other reasons such as a major fire in the port or a terrorist attack. But the truckers and shippers (not only the port) needed to have a plan in place for any significant delay.
Planning to respond to issues caused by external dependencies is important, because your customer or client is going to turn to you to resolve the problem even though you didn’t cause it (reminds me of a tee shirt slogan, “I didn’t say it was your fault, I just said I was going to blame you.”). It is equally important to recognize that a problem may not have been directly caused by your “vendor” or service-provider. In the case of the port, the problem was caused by their software developer. So the truckers were actually the victims of a risk at the second tier of their “supply chain” (a supply chain can provide services as well as material). The port was a victim of a first-tier, external-dependency failure, and the port’s problem cascaded to multiple customers creating a crisis for the port.
At the risk of “Monday-morning quarterbacking,” it seems that the port should have had a plan in place to respond to serious introduction problems – perhaps a rapid reversion to the older system. Then they could have continued operations while explaining to their software developer that some fixes were needed.
In this case, the shipping companies, the trucking companies, the individual truckers and the port should have readied themselves for “business as unusual.”
Predict.Plan.Perform
“What should we do now?” “What should we say?”