The path to the Iowa caucus meltdown was paved with good intentions.
Democratic party leaders wanted to bridge the caucus to the 21st century, keeping the core of its public and neighborly traditions while making it more inclusive and streamlined. But the technology they turned to, and its mismanagement and miscommunication, were their undoing.
Now what should have been the party's first step toward unifying behind a nominee to challenge Trump has become a stumble, spreading doubt and division, and leaving ordinary voters to wonder whether the party leaders and process they trusted has left their voices unheard.
It's been 16 days since Iowans performed their familiar democratic routine, and the results still aren't final. The state's Democratic Party chairman, Troy Price, has stepped down and, with another caucus in Nevada looming Saturday, skeptics wonder if such voting should be abolished altogether.
"Iowa went off so badly, I fear that with any new or realigned 'early state' order, we may get left out," Kedron Bardwell, a resident of Indianola, Iowa, and political scientist at Simpson University, told NBC News.
He added that the outcome of Nevada's process "is key to whether Iowa is affected or if it's part of a new discussion of the total role and representativeness and access of the caucus system."
Even now, the results of the recanvass that both Pete Buttigieg and Bernie Sanders' campaigns asked for put the two candidates in a virtual tie. The fight is over a single national delegate, which could prove essential if the race comes down to a brokered convention.
From the beginning, caucus technology seemed like a solution in search of a problem.
Virtual caucus proposal was the canary in a coal mine
After the 2016 election, the Democratic National Committee made it clear that a primary was preferred over a caucus because primaries are more straightforward and accessible to voters; seven states switched from a caucus to some sort of primary for 2020. But for a state to keep holding a caucus, the DNC required that the process be more accessible.
So, the Iowa Democrats went down a technological path, proposing a virtual caucus: give voters who could not physically be at a caucus site the night of Feb. 3 the option of calling to make their selection instead.
Iowa state Sen. Jack Hatch assembled the team that worked to develop the platform to be used in the virtual caucus, a newer technology. Hatch said the DNC was involved from the beginning.
The Iowa Democrats proposed their plan for the virtual caucus in February 2019. The DNC encouraged them to move forward with the proposal, with what seemed like ample time to the Iowa Democrats to create a sound product. Seven months of development later, the DNC Rules & Bylaws Committee unexpectedly canceled their approval for the virtual caucus due to security concerns — a canary in the coal mine of sorts for what would come in the new year come caucus night.
Hatch detailed multiple conference calls between the DNC and the Iowa Democratic Party to link up on security, adding that the coordination started to slow down in August. Reports that the DNC planned to officially deny the virtual caucus came on Aug. 30 — the date the committee received a memo from its chief security the technology officers recommending that committee cancel the approval.
Catherine Crist, the chair of the Iowa Democratic Party's disability caucus, had been working to make the Iowa caucuses more accessible for 20 years. She was devastated when the plug was pulled from the virtual caucus.
"If you make things accessible for people with disabilities, you make it accessible by virtue for everybody else," Crist said. "Disappointment is the least of the words I would use."
Despite concerns with the virtual caucus, the state party had always planned to use a reporting app in an effort to modernize the process. The DNC expressed concern about the use of technology in elections, a party official told NBC News, but never explicitly told the Iowa Democrats not to use it the app itself. So they went ahead with an app built by Shadow Inc., a DC-based progressive political technology startup.
It wasn't the first time an app was used during the caucus. One built by Microsoft and contracted to InterKnowingly was used in 2016, to address reporting issues that surfaced in 2012. It cost at least $600,000 compared to the $60,000 for the Shadow app, and extensive user and usability testing went off without an issue, InterKnowingly CEO Rodney Guzman told NBC News. But this time around Microsoft wasn't interested in resurrecting the app, which would have had to still gone through methodical re-testing, so a new developer had to be found. A Microsoft spokesman told NBC News it was focused on helping campaigns this year.
The DNC and Iowa Democratic Party were supposed to work hand in glove. By convention, the DNC sets certain standards and advises the local parties on how to do the hands-on administration of the election: running caucus sites, answering phone lines, connecting servers. In this particular case, the DNC recommended a security vendor and the Iowa Democrats partnered with Harvard Belfer Center's Defending Digital Democracy Project to develop a playbook of procedures and contingency plans
The DNC always built in an oversight role though. It could approve or deny plans, and it suggested contract language that gave the cybersecurity vendor back-door monitoring access to the technology for security testing and applying new cybersecurity and quality control measures.
NBC News and other outlets reported mounting security concerns and unanswered questions surrounding the app in the weeks leading up to caucus night, which sparked enough hesitation from the office of Sen. Ron Wyden, D-Ore. — a staunch election security advocate - for his aide to reach out to DNC Chief Security Officer Bob Lord.
According to emails obtained by NBC News provided by Wyden's office, the Senator's aide reached out to Lord multiple times in January for more information about the app and to ask if it had been audited.
Lord didn't respond until Feb. 5, after the Iowa caucus. "The app isn't ours, and the best bet is talk to the Iowa people who built it," wrote Lord in an exchange the DNC confirmed to NBC News.
Ultimately, the app itself worked fine. The seemingly endless delay in results came from a coding issue on the back end, somewhere between moving the data reported on the app to a data warehouse, and then to the quality control server before results could be posted.
Doubts among Iowans before and after caucus day
The issue that ultimately did the app in was on the back end and invisible to most users. But even the basics of downloading and using the app was a reportedly tedious and confusing process. Because the app wasn't available in the official app stores, as was the 2016 app, users had to download an additional helper app and change advanced settings.
Chris Adcock, the Page County Democrats' chair, told NBC News that the instructions to download and use the app "sucked." She said she sent a message to the Iowa Dems asking if they can write them for a "low-tech" senior citizen, the majority of the volunteers working caucus sites.
A test version of the app wasn't rolled out to precinct captains until Jan. 18, barely two weeks before caucus night. Ultimately, on caucus day itself, there were issues downloading and logging into the app. Out of 1,765 precincts, only 624 people logged into the live version of the app and 439 successfully submitted results using the technology, according to Iowa Democratic officials, or about a quarter of precinct chairs. The rest resorted to calling — waiting on hold for hours — or even texting in their results.
Adcock, along with most Iowans interviewed for this article, expressed frustration with the focus of the coverage in the reporting after the caucuses were over.
"Every caucus had a location that was big enough to accept everybody. People helped each other. You saw democracy in action," Hatch added, noting that the DNC's distancing itself just put more onus on Iowans themselves, who he says did their civic duty. "I'll go as far as to say they're trying to blame Iowans. And that's unfair; Tom Perez should be ashamed of himself."
While Perez did call the caucuses a "major league failure" by the state party two days later, the DNC points to his later comments in which he said that the national party was also in part to blame. "We all fell short," Perez said in an interview with MSNBC on Feb. 10. "While the Iowa party administers the election, we provide help, we have a partnership with our state parties. We're all in this together. We all succeed together. And when we fall, we fall together, and we fell short, and I'm here to say, I apologize for that."
The Iowa state auditor, Rob Sand, was in the room taking calls where precinct chairs ultimately chose to call in their results. While he himself was frustrated with the delays in results, calling the decisions that led to it "irresponsible," he hit back at the national media for so quickly suggesting this would be the end of the Iowa caucuses.
"Here we are in a country where those same people rightly decry the isolation that we have: the fact that we seem to be losing our social fabric," Sand expressed, "and yet here's a process where people actually get together, with all of this effort going into people listening to each other, making decisions as a community."
"And the national media says, 'Oh, this process is too inefficient for us.' Well, I'm sorry," Sand spat sarcastically. "I'm sorry for not moving at the right pace for your talking heads, but this isn't about whether or not you want to report results that night. This is about whether or not we want to have conversations in our neighborhoods about how our democracy is supposed to work."
One caucusgoer, Thomas Lecaque, enjoyed his time caucusing this month, but believes the process is "bad for democracy" — citing concerns about those who have to work or need child care and can't participate. But he's concerned that this year's fiasco will focus on the wrong pain point of the caucus system.
"I think Iowa's status is threatened — probably dead — but given that the focus is on the mechanical of reporting the caucus versus the problems of representation and participation," Lecaque said, another state "moving into first won't fix the actual issues."
Eyes are on the Nevada caucuses this weekend, where the pressure is on to see whether caucuses can still hold in modern elections - though concerns about a digital "tool" they'll be using to calculate results are rising. The DNC tells NBC News they've deployed staff to Nevada to aid in technology assistance and caucus volunteer recruitment across the board.
"My concern is that it calls into question the integrity of voting, whether you can trust the technology associated with voting in an environment in which people capitalize on those sorts of mishaps," Kathleen Hall Jamieson, who studies political communication, and is the director of the Annenberg Public Policy Center at the University of Pennsylvania.
"Particularly when you have close elections," Jamieson added, it's essential "that no one can plausibly say, 'Well we don't know who won because the technology is inadequate to answer the question.'"