infrastructure, manufacturer
Comments Off on Presentation at Fujitsu Labs of America Technology Symposium 2012

I was very fortunate to be able to attend the 2012 Fujitsu Labs of America Technology Symposium in June. The topics were varied and interesting, from emerging technologies in education to You Tube’s exponential growth.  My panel was on Intelligent Transportation Systems and how they will be driving bandwidth requirements and technology change.


The first presenter in my session was from Fujitsu Japan, and he demonstrates a very interesting model that was built using automatic vehicle locators (AVL) in taxi cabs during the aftermath of the Tsunami.  My presentation starts at 28:00, and then there is a roundtable session after.


I highly recommend spending a few moments surfing through the FLATS2012 channel because some of the presentations were fascinating.  I was lucky to be allowed to be a part of this group.

Video streaming by Ustream

hardware, infrastructure, public transit
Comments Off on No good deed goes unpunished…

A few weeks ago I wrote an update about our new Wi-Fi service onboard the COASTER Commuter Rail.  Less than 6 weeks later, negative press:  North County Times: WiFi Connections unsteady on Amtrak, Coaster.


This is not really a surprise.  I mentioned in the last post that this is a cellular technology provided by T-Mobile, and that cellular is imperfect.  I also talked about the original approach to the service versus what was pushed into production.  This is an example of how sometimes less is more – higher density in fewer cars rather than low density in all cars would have resulted in a better experience for a smaller number of customers.


Time for a new business case analysis…

hardware, infrastructure, public transit
Comments Off on Wi-Fi Service on Commuter Rail

I am very excited to say that we are finally implementing Wi-Fi service on our COASTER Commuter Rail.  This was a very difficult and plagued project.  We tried three different approaches to implementing this service over the last 5 years, two of which resulted in vendors walking away from the project.


T-Mobile is the solution provider.  Our original proposal was to implement Wi-Fi only on one car per train so that we could ensure that it provided dependable service.  We really do not have a business case to charge the customer for this service – instead it is viewed as a customer amenity, and I have to absorb the cost into our internal IT Budget.  I stand by this approach, but it was decided that we should try to cover as many cars as possible.


This is actually a difficult service to provide in our area because the train travels at more than 70 miles per hour through some isolated terrain.  All of the cellular vendors have black zones where they drop service, but T-Mobile’s solution seems the best.


One might wonder why we are going to this expense and trouble when so many people have data plans and smartphones already.  The reason is that this is the number 1 customer request from our highest dollar customers.  So we’re making it happen.

data center, green computing, hardware, infrastructure, public transit, security, software
Comments Off on NCTD IT Named ComputerWorld Premier 100 IT Leaders for 2012

We were very excited to hear today that the NCTD IT team was named as a Premier 100 IT Leader for 2012.  I say the team because while the award is for the CIO, clearly it is the work of the team that merits the recognition.  What is amazing is that our team is standing in the presence of so many other great company names:  Lenovo, Kraft, Mazda, GlaxoSmithKline, Target, Blue Cross, the State of Colorado, Waste Management, Kaiser Permanente, Vanguard, CapitalOne, Intel, Boeing… what?  NCTD is in this crowd??


Yes.  We.  Are.  That’s how we roll.


This is a little off topic, but I thought I would share a presentation I did about my NCTD IT Team for a Leadership program I am in through the American Public Transportation Association.  This shows I am perhaps a little overly-proud of my team, but they do earn that respect.


data center, green computing, hardware, infrastructure
Comments Off on Data Center Project Pays Off

Today the data center project paid off big time. Yes, for those in Southern California it was a near disaster- we had an extensive power outage that reached from Mexico to Los Angeles, and from San Diego to Arizona.


The good news for my team- the data center stayed up 100% of the time. In fact, we lost only 1 rail communications cabinet during 10 hours of an outage.


This means that our phones were up, even when cell phones were not. Our Internet was online and we could communicate with our customers. Our emergency operations center (EOC) was live and effective. I was proud to see NCTD keep bus and rail service in play hours into an outage that was so significant for the region.


More than anything, I know that without the data center project, we would not have achieved this success.


Not everything was smooth and I don’t want to overstate the positives… but I am confident without our green data center, our EOC team would have struggled that much more. So here is to investment in technology and the ability to see a ROI.

data center, energy consumption, infrastructure, manufacturer
Comments Off on Observations about the LEED Process

The LEED saga continues.  I’ve mentioned some of the challenges that we’ve faced, but today presented another interesting one.  I assumed because our purchasing specifications included statements like “systems that comply with the US EPA Energy Star requirements” that we would qualify for the Energy Star points on the LEED rating system.  Today, I learned how naive I am.  In fact, while several vendors have machines that in fact to have an EPA stamp of Energy Star compliance, they are few and far between right now.


Energy Star was a program started initially around residential power use.  As a result, most items that have the Energy Star seal are appliances or electronics in the consumer space.  A check of the EPA website shows fewer than 15 enterprise, server class machines that qualify for the rating.  So while many vendors state that they have Energy Star-compliant equipment, they do not in fact have too many machines that actually went through and successfully completed the process.


Let’s compare some from my data center as examples.  We run an HP shop (this is not an endorsement of their product or a sales pitch, just disclosure that we have them as an architectural standard).  So I have a wide variety of their equipment.  For our Microsoft Exchange upgrade, we installed Energy Star certified HP Proliant DL380 G6 rack-optimized servers.  These are currently the only series of HP machines that have the seal.  For most of my purchasing, however, I prefer HP BL460c G6 or HP BL680c G6 machines that slot into a blade chassis.  What is nice about blades is that they share components like power supplies and fans.  So this reduces the power pull, and reduces the amount of waste in the product.  So from a product life cycle perspective, they are a better choice.


In spite of our choice to generally rely on the more energy-efficient and therefore more eco-friendly choice of the blade servers, we actually cannot claim the LEED energy star credit because these servers are merely “EnergyStar Compliant” instead of certified.  This is needless-t0-say an unfortunate outcome as we are inching closer to a possible Gold certification and any point that we miss now keeps us from that nearly impossible goal.

data center, hardware, infrastructure
Comments Off on The generator part 2

I spoke today with Bob Mobach – one of my technology gurus at Logicalis – about my concerns about the non-green generator.  After much discussion and research, we have again validated our decision to move forward with the generator.  He had some excellent insights about why we are in this position, and when some of the alternative technologies are more appropriate.


NCTD requires a high-availability environment now.  There is no getting around the fact that we run important, mission-critical systems on our infrastructure, and as such we can no longer afford to have unexpected and unplanned outages.  To meet this basic requirement, we must have the capability of creating power on-demand, nearly instantaneously should our power fail.  There simply are few choices on the market today to address this need for a facility of our size.


We could have taken the approach that we would provide our own ‘co-lo’ power via a set of natural gas generators.  This scenario would have allowed us to failover to SDG&E power should the natural gas units fail.  This option does not really make sense for our data center because firstly we’ve invested in solar pv power to offset the power demands, and secondly because our data center power draw is really too small to make this economical.  So from a business case, this is a poor choice for our scenario.


Another option was the fuel-cell based UPS.  The problem with this approach is simply the cost and the durability of the power.  We have had now two outages this year over 6 hours in duration.  There is simply no way we could sustain the entire data center on the fuel cells for this long.  And again, the business case is not there.


We are now investigating alternative fuel sources for the Kohler generator – either ‘clean diesel’ or biofuels.  While the biodiesel seems like a natural choice, the biggest issue is the long-term storage of the fuel.  Fuel that sits around for months can degrade due to algae or condensation.  Research indicates that not many entities are yet using biodiesel as their fuel choice for standby scenarios where the generator is not going to be used frequently.  So my next step is to contact the manufacturer to determine whether a biodiesel mix is appropriate in the generator, and what they would believe the ideal mix to contain.  We will then compare this to the emissions, performance, and price of ‘clean diesel.’


So while I am still not 100% reconciled that installing a generator is moving us in the right direction for a green data center, I am convinced that we’ve made the business decision which balances our environmental needs with our business realities.

data center, green computing, infrastructure
Comments Off on Evaluating Backup Power Strategies

One focus of our data center design activities this week has been around the question of backup power. Any high-availability data center must face this question, and the reality is that there are not too many options to have a green approach to backup power.


The first consideration is the Uninterruptible Power Supply (UPS). We looked at three options for providing the immediate, on-demand UPS capability: traditional battery UPS, flywheel UPS, and Fuel Cell UPS.


The flywheel UPS seemed to provide the greenest option; however, it was not available to integrate with our chosen ‘pod’ manufacturer. This leaves us with the fuel cell or the battery options. While the Fuel Cell is a solid option for the future, I felt that the cost/benefit was not yet there for us. Which leaves us with a traditional battery for our UPS. Not the greenest option overall, but when you factor in the full APC Infrastruxure solution, we still come out ahead of the game.


The second major decision was on backup power to support the UPS for extended potential power failures. Most SMB data centers do not really face this issue. However, in the last two years of my tenure, we have had no fewer than three extended power outages that have taken down my data center for over 8 hours. Now that we’re running the SPRINTER train and the fare collection systems using the network, extended downtime is simply not an option.


We are therefore going to install an onsite generator. I initially was not happy with this idea because clearly a diesel generator is simply not green. We were initially pursuing a natural gas generator under the assumption that this would be a more environmentally-friendly option. However, in working with our generator installation firm (Bay City Electric), it became clear that natural gas would not be an option for us due to potential issues from earthquakes interrupting the supply lines.


So we’re going with the traditional diesel generator unless some other option presents itself. Our strategy will be to mitigate as many environmental concerns as possible through process:

  • Whenever possible, we will test the generator under load -meaning that we will run the data center 100% on the power the generator creates. This means that we will not be wasting the fuel.
  • We will install filtration on the emissions to ensure that we’re not discharging significant particulates
  • We will test as infrequently as possible to validate that the system in functional


I spent hours working with our IT firm Logicalis to try to find alternatives to these two decisions. I have to say that I am disappointed that there were not other options readily available in the market place for us. In reviewing some of the most successful green data center project case studies from the last two years, the preponderance of them make reference to their use of generators for backup power. Even my green-web hosting firm talks about their use of generators in times of need.


While we are making great strides on designing our green data center, I have to admit that I was disheartened with these two decisions I made in the design; however, I did not feel I had the business case, other case studies, nor the viable alternatives that would lead us down a different path.

data center, energy consumption, green computing, hardware, infrastructure
Comments Off on Data Center Redesign Kickoff

by Angela Miller
On Friday we kicked off the data center project at NCTD, and rarely have I been so excited about the probable success of a project.  Our vendors are Roel Construction (Rob Netzer) and Logicalis (Bob Mobach).  We were lucky to find vendors that have such competency and experience in data center design, and specifically in the requirements for attaining LEED certification.

This is a large undertaking for a small entity like NCTD.  While I can justify the project purely on the long-term anticipated Return-on-Investment, the deal was sweetened by meeting all of the requirements for the Federal Stimulus program.

The overall project has several elements that will hopefully qualify it for LEED certification, including:

  • The anticipated reduction in power demand and increase in power utilization efficiency in the facility
  • The re-use of the building, and materials within the facility for the project (for example, we are supplementing our green fire system instead of replacing it, we’re going to reuse doors instead of purchasing new ones, etc.
  • The ability to reset the ambient temperature of the data center much higher – we expect to set it around 80 degrees instead of the 68 we maintain today
  • The installation of over 220 Kw solar system onsite to more than offset the power draw of the data center
  • The use of in-line cooling instead of the two air handling units currently in the room (these will be recycled for other purposes in the District)

In addition, as I’ve mentioned in previous posts, the District has been committed over the last two years to slowly introducing greener, more sustainable approaches to our information technology infrastructure, including virtualization and consolidation, switching to blade server technology, replacing older equipment with more energy-efficient equipment, and testing desktop virtualization.

All of these steps make solid financial sense for the District – an especially important consideration given the tight financial times we are currently facing.  I could not in good conscience recommend these investments if I could not show solid ROI for our troubles.  While we want to be an agency with a priority on sustainability as part of our mission, it is logical that we could not choose to make these investments if they required a corresponding reduction in service or positions at the District.  Instead, choosing to follow the more sustainable path will realize direct operating cost savings on a monthly basis for the District.

We of course face some challenges to our ambitious timeline.  One of those challenges is the delay of a system migration project by 90 days after its anticipated completion.  This project is attempting to move our Prime System applications (installed in 1986) from a minicomputer to a client-server web interface.  We cannot risk downtime on this environment and therefore out-of-the gate may see a 90-day delay.  But every IT project faces challenges, and I am confident the team will find an approach that gives us what we need.

Over the next several weeks I will be blogging about the adventures of upgrading our data center.  Hopefully in March I will be able to say that we’ve completed the work and that we are on track with our ROI.