Design Elements of Performance

An equipment spec used to look like the Yellow Pages (and sometimes they still do). Part of the reason behind this was that agencies dictated down to the nuts and bolts, literally, how their equipment was to be designed. Sure, this gave them control over the process, but this spec micro-management also left them responsible should anything go wrong with their equipment. It allowed suppliers to say, “Well, we did what you asked us to.”

Performance Base
In an effort to provide the best equipment possible for their customers, many agencies have shifted toward or wholly adopted a performance-based spec process. With a performance base the spec states what the agency wants the equipment to be able to do, not exactly how to do it, allowing the supplier to determine in their opinion the best way possible to accomplish this task.

“For vehicles, for just about anything, we try to do performance based. That’s not to say if you look that 100 percent of what we do is performance based, but that’s preferred,” says Andrew Murphy, equipment engineer for Austin’s Capital Metro.

“I guess it’s sort of like, I’m not in the business of designing a bus. I mean, I could be, but I don’t know the latest technologies, I don’t know the most recent materials that are out there, I don’t know the most economically advantageous manufacturing techniques.”

Murphy explains that in a recent process to purchase some paratransit vehicles, he was looking at getting an air conditioner that would cool the bus sufficiently. Rather than tell them exactly what he wanted, he just wanted it to cool the bus. When asked how many BTUs he wanted, Murphy said he didn’t know, he just wanted it to cool the bus, explaining that if he required a certain amount of BTUs and was wrong, he could have a vehicle that fit his spec, but not do what he wanted it to. By giving a performance-based spec, there is an implied warranty that the equipment will perform as the agency has requested.

“If the air conditioning doesn’t cool the way I want it to, [the supplier] could point out that they put everything physically into it that I told them to. And they put the screws in where I told them to and everything. But then when it doesn’t cool, I have no recourse. Here if there is a problem, if it’s with an air-conditioning system or some other mechanical component, the warranty is on the performance, if you would, in addition to the individual components. In other words the bus needs to cool, the bus needs to run or the bus needs to start or do whatever it needs to do.”

Murphy says that he has seen a significant change in the last five to eight years in the way his agency has done speccing. Where they used to use the tried-and-true design-based philosophy, now they are almost entirely performance-based. Why?

“I think there’s been a variety of factors and the most obvious might be that there’s nothing worse than having a problem with your bus and realizing that it was built to your specification, but that your specification was maybe wrong,” says Murphy. “Or that the people that wrote the specification at the time didn’t fully understand what it was that they were writing.”

Murphy says that many design-based specifications actually come from the suppliers themselves. Agencies then copy the ones they like into their specs, but human error can crop up and cause an important part to be left out. Using a performance-based spec, the agency can simply say they want a product to last a certain number of years and be resistant to wear and let the supplier figure out how to accomplish that.

Murphy says design is difficult as you can easily overlook something, whereas with performance, “…it is sort of a black box thing, I just want to put something in this side of the black box and I want this thing to come out of the other side of the black box, I don’t want to have to worry about what happens in the middle there.”

So how does performance-based spec design benefit an agency? In a word — accountability.

“I think it provides, for the agency, more of an accountability from the supplier or the manufacturer or whether it’s a vehicle or a component of a vehicle. It sort of puts the onus on them to deliver something that is going to work for the agency and ultimately for the public,” says Murphy.

“That’s the biggest thing. In simpler terms you just get together, you shake hands, you say does this really do this, that’s great we have it in writing. Now if there are any problems, we can go back and say, you said it was going to cool the bus, you said it was going to go 100,000 miles. And then the onus is more on them as to try to explain what they’re going to do to make it do that. As opposed to pointing toward some line on the specification that says, well you know we did everything that you wanted or everything that you stated on your specification.

“I think the benefit it has given us is a more reliable vehicle or a more reliable component because it forces the manufacturer of that component — or it allows is another way to look at it — it allows the manufacturer of that component to design it to the best of their performance ability as opposed to just meeting the designs we’ve stated.”

Industry Standard
Capital Metro’s move away from a design-based speccing process toward a performance-based one could almost be seen as the opposite end of the spectrum. According to Brian Iacono, Denver RTD’s senior manager of materials management, his agency has stuck to what could only be called the Industry Standard.

Iacono says that while he deems their spec process a combination of design and performance base, it is largely written to the specifications laid out in the American Public Transportation Association’s (APTA) “White Book,” or its Standard Bus Procurement Guidelines, which it designed in partnership with the Federal Transit Administration. These guidelines present a “standard, complete bus procurement package” that can be used both for RFPs (Request for Proposals) and IFBs (Invitation for Bids). Iacono says that RTD’s specs also have to take into account Denver’s environmental conditions, such as high-altitudes and weather-related concerns.

“It’s pretty detailed,” says Iacono, “but it speaks to the White Book quite often. And it also speaks to a lot of industry standard requirements whether they be, for seats for instance, the Federal Motor Code vehicle requirements for flame retardant, that type of materials.

“So there are a lot of references to industry standards in [our specs]. And, of course, several references to the White Book specs in here as well.”
So how has RTD benefited from this process? According to Iacono, it has allowed them to reference things like SAE standards and consider them industry requirements, while at the same time giving them a flexibility to input into the spec based on their experience.

“We really start out with trying to adhere to the White Book spec and that’s primarily what we do,” says Iacono. “But there are, based on our experiences, and not only just dealing with the local environment, but there have been some experiences where we want to stick to a certain design on something.”

Cooperative Process
While other agencies see the spec process as a chance for them to tell suppliers what they are looking for in new equipment or what they want the new equipment to do, Bob Baulsir, Nashville MTA’s chief operating officer says for his agency it’s all about the negotiation process.

Baulsir says that MTA uses the RFP process and that leans them toward performance instead of design as far as specs go, but it really comes down to meeting with the vendor.

“Although in the RFP process we might say, hey we’re looking for this and looking for that, then there’s that negotiation that follows. And what happens in that process, at least with the RFP process, we may meet with the vendor and they’ll say we can’t do this and this is why. And then we’re sitting across the table from each other and we’ll say specifically do we need that, no, here’s our goal, here’s what we’re trying to accomplish,” Baulsir says.

Using fuel tanks as an example, Baulsir explains that while you may begin talking about minimum sizes, you end up discussing what’s needed for the buses to run for 18 hours.

“The process probably should start at absolute performance, but it doesn’t, it’s more of a negotiated process where you end up with, OK here’s what we need, how can you do it?

“Where we’re not necessarily technically designing what they’re doing, but in the end it just meets the performance figures we are looking for whether it’s distance or climate control or something like that,” says Baulsir.

Baulsir says that while the agency usually starts with the old “we design it” piece, it comes down to sitting down with the vendor and finding out what the agency is trying to do and finding out that the vendor knows how to do it.

“Sometimes what you find with a particular application, let’s say it’s an engine and we’re asking for a specific piece on that engine, that this has already been tried with that type of engine and it doesn’t work. And you get a chance to talk about it and they say, hey we did this for somebody who pretty much had to have it and it didn’t work and we had to pull them all off.

“And you have to sit there and listen to them and say well, this is good information, maybe we need to reconsider, maybe we don’t need this this way. And let’s listen to the folks that are building these vehicles.”

Conversation is the backbone of a good procurement according to Baulsir, “It seems to me that the key to the whole thing is the RFP process where you actually have open dialogue.

“Where you might start out saying, hey this is what we need, and on the other side of the table they are saying, now wait a minute, you know here’s why you don’t want that.”

In the end, what are the benefits of this open negotiating process for the agency? Baulsir says it saves them money by not having to increase staff.
“I think the benefit, especially for an agency like Nashville, we’re not big enough that we could ever imagine that we could design a complicated piece of equipment. We don’t keep engineers on staff and all of those sorts of things.

“You know, we’re looking toward the vendors to give us that expertise that we wouldn’t have. If we were one of the larger systems in the country we might be in a position where we would attempt to engineer our own product, but we just don’t have those types of resources internally to do that.”