green computing
Comments Off on Changes

Well it is official: I am no longer with the North County Transit District. Looking over the last five years, I can say this has been a grand adventure with a highly-skilled technology team. We did some great work for the people of San Diego County. From security, to sustainability, to customer applications, to Green IT – we had the chance to innovate. I am extremely proud of the work this techie superhero team performed. What a ride.


But change is good and I am excited about the future and the opportunity to do work for a variety of companies. The ecology.IT consultancy is up and running and ready for engagements. And there are some interesting blog notes that will be coming soon: a trip to Germany to view energy-efficient transit technologies, Oracle OpenWorld, and APTA Annual and an investigation of emerging transit technologies. Stay tuned for further adventures.

data center, green computing, hardware, infrastructure, public transit, security, software
Comments Off on NCTD IT Named ComputerWorld Premier 100 IT Leaders for 2012

We were very excited to hear today that the NCTD IT team was named as a Premier 100 IT Leader for 2012.  I say the team because while the award is for the CIO, clearly it is the work of the team that merits the recognition.  What is amazing is that our team is standing in the presence of so many other great company names:  Lenovo, Kraft, Mazda, GlaxoSmithKline, Target, Blue Cross, the State of Colorado, Waste Management, Kaiser Permanente, Vanguard, CapitalOne, Intel, Boeing… what?  NCTD is in this crowd??


Yes.  We.  Are.  That’s how we roll.

data center, energy consumption, green computing, public transit, solar, sustainability
Comments Off on Press Article on NCTD IT

nctimes-millerAs much as I am not happy with the photo, I am happy that the North County Times decided to feature my team in an article today.  The focus is on our sustainability and technology programs.  Thanks to Paul Sisson for the kind words.


In the background is our new solar installation at the Buena Creek SPRINTER station.  This is an exciting new technology by Uni-Solar called “photovoltaic laminate”.  Basically it is a flexible solar membrane-like material that can be glued down to the supporting structure.  It is highly durable – used by the military in harsh installation conditions.  It is not as efficient per square foot as other technologies, but we chose it to prove that solar can be installed in some pretty challenging environments.  You can read about the material here:  uni-solar_laminate.

Photo courtesy taken by Hayne Palmour IV

data center, green computing, hardware, infrastructure
Comments Off on Data Center Project Pays Off

Today the data center project paid off big time. Yes, for those in Southern California it was a near disaster- we had an extensive power outage that reached from Mexico to Los Angeles, and from San Diego to Arizona.


The good news for my team- the data center stayed up 100% of the time. In fact, we lost only 1 rail communications cabinet during 10 hours of an outage.


This means that our phones were up, even when cell phones were not. Our Internet was online and we could communicate with our customers. Our emergency operations center (EOC) was live and effective. I was proud to see NCTD keep bus and rail service in play hours into an outage that was so significant for the region.


More than anything, I know that without the data center project, we would not have achieved this success.


Not everything was smooth and I don’t want to overstate the positives… but I am confident without our green data center, our EOC team would have struggled that much more. So here is to investment in technology and the ability to see a ROI.

data center, energy consumption, green computing, sustainability
Comments Off on APTA Presentation on Building a Sustainable Data Center

I’ve uploaded my APTA presentation about building a sustainable data center to slideshare if anyone is interested.  Clearly NCTD is a pioneer in the transit industry, even if building a sustainable data center is cliche in other industries.  Not one participant in the room had added sustainability as a design criteria when building their data centers.  We’ll see if I made any impression with the community on this issue.

data center, green computing, sustainability
Comments Off on Materials Posting #2: Toilets, Taps and Trees

photo courtesy this is our selected toilet for the datacenter.

Well today was an interesting meeting with our LEED consultant Brandon Smith.  Based on our meeting, it is clear that specific LEED requirements for data centers do not yet exist.  As a result, we’re pursuing a classification for an Interior Space (for reference, the requirements are here).  There are a few items from this list that simply do not apply to data centers, but are ‘gatekeepers’ that must be addressed in order for us to pursue any certification.


The first of these is that no part of the interior space up for LEED certification can be cooled by a CFC-based air handling system.  Unfortunately our building was constructed years before the non-CFC requirements came into practice and therefore the majority of the facility is in fact in violation of this requirement.  However, we had previously installed two CFC-free air conditioners dedicated for the data center.  Our original plan was to reuse these air conditioners to supplement the cooling in our board room; however, given the LEED requirements we are now going to use one air conditioner for the ambient air handling in the Data Center commercial space, and the other for the Board room.  Had we not possessed these air conditioning units, this would have been the end of pursuing LEED for us.


The second set of requirements that are unforeseen gatekeepers are those around reduction in water use.  Interestingly, we use absolutely no water in the data center as none of our equipment is water-cooled.  We made the mistaken assumption that no water use would be considered a good thing.  We were wrong.  As a result, we are now forced to add in the public shared spaces on the floor for consideration in the commercial space, and then to show not only a reduction down to the required baseline water usage, but then an incremental reduction from the baseline.


Again, this is an older facility, so we have older toilets and water fixtures.  So Mr. Smith is now working on the 5 toilets, 2 urinals, and 5 water faucets we will need to replace to be considered for certification.  I am wondering how the price will impact my overall return on investment calculation.  The cost for the LEED certification itself is approximately $15,000, and now the incremental cost of the water use reduction could be an additional $5,000.   I will refrain from discussing how this is a crappy situation.


To answer this question, I made the decision that logically we would want to spend no more than 1 advertising campaign would cost.  This would be the net cost – in other words, if the LEED portion of this project costs us $20,000 to go through the process, and if our average advertising campaign costs us $5000, then in order to have a $0 incremental cost the investments through LEED would need to save $15,000 through their total lifetime in order to justify the expenditure.  My logic here is that by successfully obtaining an LEED certification for a commercial interior space – the first such certification for NCTD, and for northern San Diego County – I would likely be generating some press and attention through the investment perhaps equivalent to one small marketing push.  The rest of the investment must be justified by some other tangible return.


Brandon and I are working on this question now.  Just for edification, here are the other items we’re considering in order to meet the basic LEED certification requirements:

  • Designating some of our parking spaces for carpools or vanpools
  • Reusing our interior door from the project instead of purchasing a new one
  • Measuring the Solar Reflective Index of the concrete around the building
  • Measuring the shade of the trees on the parking lot


To be fair, I understand the need to consider the overall building in this project.   Given that this is a ‘green data center’ that we are building inside of an existing and rather dated facility, the task of addressing all of the needs of LEED may be insurmountable.  Especially considering my desire to also establish the business case for the project beyond the tangible benefits of the data center itself.  In other words – building the case for green instead of conventional data center practice.


Once we have finished the complete ROI analysis, I will post.  Until then, dual flush or low flow?  That is the question…

data center, green computing, renewable materials, sustainability
Comments Off on Materials for the Data Center: Floor and Ceiling

Now that we’ve faced down the crises from our various storms these last two weeks, the team can again turn attention to the final elements of planning around the NCTD data center project.  Our current data center has an elevated floor and a standard office dropped ceiling; however, the physical constraints of the room prevented installing either of these at their recommended heights.  The raised floor is very shallow, and does not provide enough space to be utilized for air handling.  In fact, the primary original purpose of the raised floor was to allow for piping for the fire suppression system.


The floor has no routing or conduit for cabling, resulting in a haphazard approach to cabling in the room:  some racks have cabling in ad-hoc runs constructed above racks, while other systems have random cables under the floor.  This resulted in a relative rats’ nest of cables discovered as we temporarily relocated some of the systems this month.


Originally we investigated re-using the raised floor; however, to accomnmodate the growth in equipment we desired to extend the room by 2.5 feet resulting in the need to procure raised flooring for another 50 sq ft.  The original manufacturer does not exist anymore, and the product was not something we were able to find on the open market.  I am frankly happy that we were unable to locate a suitable product to go with this system, as it is simply not ideal for our situation.


This work resulted in the team deciding to remove both the existing ceiling and flooring systems.  Our installation approach now is to instead install electrostatic dissipative (ESD) tiles on the subfloor, and a new dropped ceiling system that will provide for a more logical approach to cabling, air handling, and fire suppression.


My first stop in researching materials was again the US Green Building Council website, which described a program from the Resilient Floor Covering Institute(RFCI) called FloorScore.  Much like the scores provided around sustainable forestry processes, the RFCI FloorScore provides guidelines and a mechanism for an independent assessment (through Scientific Certification Systems) of the volatile organic compound (VOC) emissions from the materials.  Similar to the VOCs from paints, floor and ceiling tiles – and other industrial products – can come with a large VOC emissions load.  From both sustainability and worker comfort perspectives, choosing products with low VOCs are preferable where possible.


In addition to low VOC, we’re requiring that the selected products contain a substantial percentage of recycled or renewable materials.  We also would like to work with a company that will allow us to recycle all of the flooring and ceiling material from the existing room.


Surprisingly, there are now a wide variety of products on the market that will meet all of our needs, from ESD, to material composition, to recycling services.  My original fear that was in requiring the additional environmental criteria, we would be facing a substantial increase in the project cost.  However, that has not proven to be the case.  Instead we’ve discovered a wide array of companies that have embraced the need for these products and who are delivering a variety of choices to the marketplace.


We have selected a flooring product manufactured by Armstrong for our project.  Not only do these products meet all of the above criteria, but one of their manufacturing facilities is within 500 miles of the NCTD project site.


The purpose of this posting isn’t to market the specific product we’ve selected, but to instead point out that it was possible for us to impose additional purchasing criteria on the project, find products that met the criteria, and to do so in a manner that did not substantially increase our costs.


Our purchasing criteria included:

  • Must meet all anti-static or static dissipative requirements for a data center
  • Must contain at least 15% recycled materials (would prefer higher percentage)
  • Must be FloorScore certified
  • Desirable to be sourced within 500 miles of the project site
  • Must have low VOC emissions

Dig Deeper


There are a wide array of manufacturers who provide flooring and ceiling products designed specifically to address both the needs of a data center and the environmental purchasing criteria we selected.  Rather than providing product links, instead I would encourage that people review the RFCI and USGBC sites for information about establishing the appropriate criteria for your project.

data center, green computing

This week we started investigating the LEED certification process and it was illuminating. For my data center project we only have two options for obtaining the certification- and both are frankly uphill battles.


The first option is for an interior commercial space. Our LEED consultant Brandon Smith believes we may be able to qualify under this program at a basic certified level. Our data center is part of a shared use space in the basement of our building. We lease out the third floor of our facility to at least three other companies. We also share a lunch room in the basement, and lease out another office in the basement to the same companies.


The data center is a separately-accessed space that is one small office on it’s own. I think this is our best shot at certification. Mr. Smith believes our project will garner 47 of the 100 possible points, which would put us right at the certified level.


Our other option is to pursue an Existing Facility Operations and Mainentance certification. This is a more rigorous review that would require us to change out our old whole-building air conditioning units. I cannot see how I can justify this large capital investment on the data center project alone.


We will investigate the possibility of the second certification as part of our larger sustainability plan.


There were some surprises in this evaluation:

  • The rather large solar investment in the project- enough to offset 100% of the power needs of the data center – garnered us all of 1 point in the evaluation. The same as reusing our door.
  • We received the bulk of our points because of the location of the site and the nature of our business. Being a public transit district and the free transit we offer out employees brought us 6 points. The thought is that commuters will generate a much larger carbon footprint than that of the data center itself.
  • We receive no points for reusing the air conditioners. Even though they will be used infrequently in another part of the building, they clearly still pull a great deal of power.


Pursuing the second certification will require significant investment on our part- from changing the toilets and water fixtures, to the cleaning products we use, to changes in out parking lot. It will be an interesting process to determine whether these investments will make sense to our financially strapped organization.


And that brings us to the question of ROI. The certification process will probably cost around $7500 with fees and consulting hours. This is a very small cost on the overall project, but is equivalent to purchasing 3 servers. Will it return as much as 3 servers would? Is there a monetary value to LEED?

data center, green computing, infrastructure
Comments Off on Evaluating Backup Power Strategies

One focus of our data center design activities this week has been around the question of backup power. Any high-availability data center must face this question, and the reality is that there are not too many options to have a green approach to backup power.


The first consideration is the Uninterruptible Power Supply (UPS). We looked at three options for providing the immediate, on-demand UPS capability: traditional battery UPS, flywheel UPS, and Fuel Cell UPS.


The flywheel UPS seemed to provide the greenest option; however, it was not available to integrate with our chosen ‘pod’ manufacturer. This leaves us with the fuel cell or the battery options. While the Fuel Cell is a solid option for the future, I felt that the cost/benefit was not yet there for us. Which leaves us with a traditional battery for our UPS. Not the greenest option overall, but when you factor in the full APC Infrastruxure solution, we still come out ahead of the game.


The second major decision was on backup power to support the UPS for extended potential power failures. Most SMB data centers do not really face this issue. However, in the last two years of my tenure, we have had no fewer than three extended power outages that have taken down my data center for over 8 hours. Now that we’re running the SPRINTER train and the fare collection systems using the network, extended downtime is simply not an option.


We are therefore going to install an onsite generator. I initially was not happy with this idea because clearly a diesel generator is simply not green. We were initially pursuing a natural gas generator under the assumption that this would be a more environmentally-friendly option. However, in working with our generator installation firm (Bay City Electric), it became clear that natural gas would not be an option for us due to potential issues from earthquakes interrupting the supply lines.


So we’re going with the traditional diesel generator unless some other option presents itself. Our strategy will be to mitigate as many environmental concerns as possible through process:

  • Whenever possible, we will test the generator under load -meaning that we will run the data center 100% on the power the generator creates. This means that we will not be wasting the fuel.
  • We will install filtration on the emissions to ensure that we’re not discharging significant particulates
  • We will test as infrequently as possible to validate that the system in functional


I spent hours working with our IT firm Logicalis to try to find alternatives to these two decisions. I have to say that I am disappointed that there were not other options readily available in the market place for us. In reviewing some of the most successful green data center project case studies from the last two years, the preponderance of them make reference to their use of generators for backup power. Even my green-web hosting firm talks about their use of generators in times of need.


While we are making great strides on designing our green data center, I have to admit that I was disheartened with these two decisions I made in the design; however, I did not feel I had the business case, other case studies, nor the viable alternatives that would lead us down a different path.

data center, green computing
Comments Off on Designing the cooling system for the NCTD Data Center

by Angela Miller
One of the biggest issues for my little data center at NCTD is handling the heat load of the room.  I was reading an article online at the Georgia Institute of Technology that said that cooling the data center has become more complicated as the average heat load per cabinet has moved from  1-5 Kilowatts of heat to 28 in the last 5 years.  We can easily see this in our room – we have a cabinet with 10 rack-optimized HP DL360 servers each with a 1 unit space between for air flow sitting next to another rack with a C3000 8-server blade chassis and a SAN sitting next to a rack with a C7000 16-server chassis with the Cisco VOIP equipment and no spaces between servers.  This little example shows how in just 5 years the density of the typical server rack in our room has increased immensely.

We also have first-hand experience with the problems this increased heat load can cause for the facility itself.  While we have a raised floor in the room, it was designed primarily for cable management instead of heat mitigation.  So the floor tiles are solid and do not allow the cooler air to be funneled through to the cabinets.  About 3 years ago the heat in the room exceeded the capacity of the air conditioners resulting in both a flood of condensation in the room and a blown ac that took down the data center.

I have previously blogged about our current poor cooling solution installed as a result of this outage – two residential-class air handling units on the floor of the data center with a fabricated venting system designed to pull in the hot air from the floor (?) and push the cold air from the top vents directed at various angles throughout the room.  This system makes it extremely uncomfortable for my staff to work — and who can blame them?  I myself moved one of the vents just an inch higher so I could stand in the room for an hour and forgot to move it back.  This one little adjustment resulted in the temperature in the racks increasing on average 3 degrees while the vent was moved.

Clearly this is not a sustainable solution.  So as we embark on the redesign of the data center, cooling solutions have been front-and-center in the conversation.  The Green Grid has published seven steps to consider when designing a cooling solution for the green data center:

  1. Developing an air management strategy
  2. Moving cooling systems closer to the load
  3. Operating at a higher delta-T
  4. Installing economizers
  5. Using higher-specifi cation and performance equipment
  6. Using dynamic controls
  7. Maintaining higher operating temperatures

We kept these guidelines in mind when evaluating the options for cooling the data center.  We have looked at a variety of cooling solutions for the facility, but there are several things which make it difficult to be innovative in this space.  The first is that this room is in the basement of a former bank building.  We are strictly limited on the height and footprint of the room.  Therefore we cannot be more creative with our raised floor – it is only 8 inches high, but going higher to allow for venting under the floor is not in the cards.

We also must contend with walls that support vaults on two floors above the data center, limiting what we can do with the venting and air handling outside of the room.  Given these constraints, the recommendation by Logicalis and Roel was to install a pod system.  This approach will allow us to encapsulate the racks, create hot and cold zones, and provide in-line cooling for the racks right where the need is.  This is not ideal for every data center – for example, we almost were unable to use this solution because the footprint of the room was 1 foot too short for the necessary clearance around the pod.  Fortunately, we were able to recapture some space by moving an internal wall out slightly allowing us to just fit the equipment into the design.

It is also important to understand the tradeoff with an encapsulated system like the APC pods:  once it is in within the walls of this room, I will not have the ability to grow the data center past this size.  This will be the finite number of racks we can install in the foreseeable future.  I cannot move walls again, nor can I migrate to a new space within this building.  So we must be smart in the design phase in order to get us a full ten-year investment and growth opportunity in this space.  Choosing a pod also increases the budget versus sticking with the current approach of open racks in the space.  But given the other design criteria, the pod solution is the clear winner on energy efficiency and heat handling.

While this is the basic design we’ve elected to pursue, I also requested that we begin by first performing an air flow analysis of the current facility, both with the air conditioners running and without.  Such a study might reveal some interesting design criteria for us to keep in mind as we move forward.  I have a feeling that we might find we have significant bypass airflow issues to deal with (basically this means air that is infiltrating the room through gaps and openings in the walls).  Our initial monitoring of the humidity in the room using simple environmental monitors shows that we actually have a moisture problem in the room in addition to the cooling issues.

I will post more as we get into this project so that you can see the practical realities of how we make decisions that consider the energy efficiency, sustainability, and practical design considerations through the project.

Digg Deeper on the Issues:

I relied on the following sites for this post:

The Green Grid
American Power Conversion

Georgia Institute of Technology

None of the entities in this post has provided compensation or incentive to discuss their products or services.