David Laube: Failure To Use Openstack

In the early summer of last year, my colleague Zac, also CEO of the company, asked me for help to build a modern cloud hosting platform with nothing installed.

I can't help thinking about my past experience, including the experience of building, supporting and using extensible infrastructures.

I asked myself, do I really need to do this? Are there many good Infrastructure as a Service (IaaS) that can be used?

With the deepening of communication, I finally realized that many cloud services are not user friendly, and there are great difficulties in using them.

In addition, I am an early user of Docker, and Docker is an application container engine. The deployment plan supported by this container will make the high quality physical bare machine more powerful in operation and maintenance.

But some public cloud virtualization, and some of the problems of hosting providers, have not been able to match the needs of complex and varied physical hardware development.

So I feel that I need to do some work for this.

Next, let's get together with packet.net's deployment journey.

I plunged into the work of deploying packet.net.

At the same time, we are busy focusing on the deployment strategy and the relevant developments in cloud automation. From the beginning to the end, we examine specific installers, all the open source cloud platforms, and the services we have installed.

Voxel is a cloud hosting hosted platform bought by Internap. We deployed a lot of our own programs when we used it. In the process, we saw both the benefits and the feeling of having a software platform.

The installation of the server seems to be very easy. It seems that once completed, once and for all, right? But it is absolutely false! Because after installation, there will be countless network problems, as well as the adjustment of hardware at any time, as well as the differences of various operating systems.

In such a situation, it can provide users with the best.

To install and manage thousands of servers and ensure that these servers work properly, they can respond to decisions made by Zac within five minutes.

This is not an easy thing for me.

In order to make packet.net arrive

Thousands of servers have been installed and started 7x24 hours in a few months.

I began to focus on the uniqueness of OpenStack in the Internet infrastructure, which can be used as a means of building services.

This includes the automation of the networking business, the management of the IP address, the monitoring of the installation process, and the replacement and installation of the hardware.

If I can rely on OpenStack's core projects to complete the work, then my team will be more focused on things that can bring more value to users, such as hardware analysis, and also provide technical support for the application engine of container mechanism.

I was reminded of some of the hidden dangers of OpenStack, but I spent several weeks reading the latest version of the record, mixed up with several Wiki's IRC official chat channels, and played the OpenStack installation script DevStack.

I started right.

The core project is no longer so strange.

In the past two years, DevStack has developed very well, and the timing is just right.

Rackspace, the world's leading managed server and cloud computing provider, has recently released the OnMetal physical bare machine server deployment plan, and has written blogs publicly about how to deploy Ironic on its physical machine.

In October 16, 2014, an important version of OpenStack, the Juno version was also released.

So I think OpenStack should be used to deploy the physical servers of the company.

I know that the process of learning OpenStack will not be smooth, and I understand that it requires hard work to learn every project, not just installation.

I studied every item of OpenStack in detail, trying to understand the dynamics of Nova, and the driver of Ironic, especially Neutron.

We not only install Ironic on the physical server, but also support the network model of packet.net hosting service, especially using Layer3 to replace the functions of Layer2 and VLAN layer hosts.

At this time, you may say, "Hello, there are so many documents to read and learn"! In the past month, I obviously feel that the documents we contacted are not outdated or wrong.

This makes me have to delete content from previous high-quality documents, such as articles from wiki, IRC (a chat tool), and version submission records, from these places to find the latest correct information.

After these basic work is completed, I will use Python to do a lot of debugging work to verify various functions inconsistent with document description.

For example, whether this work is right or not is a long process.

It is worth mentioning that there are so many people and companies who rely on OpenStack to survive and form a great symbiotic system, especially the part of OpenStack's Nova and standard Neutron projects.

Although this group can compete with other open source projects on a large scale, it is difficult for Ironic to achieve product level usage.

I encountered such a situation. I consulted some core developers about some implementation problems, but they could not answer them.

And I search these questions from Google, and I can only get a handful of information about problems.

I gave the Neutron part to my colleagues, and I got to know Ironic better.

But the reality is that we need every part of the OpenStack developer to help us understand the code base so that we can keep pace with the update of every OpenStack project.

Then how do we go to meet our needs properly? So I contacted IRC members from the OnMetal team from Rackspace and contacted by e-mail.

Go to the OpenStack Developer Forum.

I dare to make sure that I read every relevant document, and every post in the forum, and I also debugged Ironic through the relevant information from Google.

Although there has been a breakthrough in the OpenStack Nova version of the physical server deployment scheme for previous Ironic projects, OpenStack has been designed at the core of virtualization technology.

Many functions and document modifications still exist between Nova's physical machine deployment plan and the Ironic deployment plan with driver.

I feedback this situation to the limited Ironic technology support department, but I am hard pressed to use openvswitch and linuxbridge related to virtual technology.

Our network model has a serious conflict with this.

So I found that OpenStack's Neutron project not only lacks technical support for specific network product vendors, but also lacks the ability to expand different network models.

A large user with a deeper understanding of the core code of OpenStack (the typical Rackspace company) relies on highly customized OpenStack projects to enable them to deploy physical machines on the physical network.

Several patches have been released, but many important patches are not published, requiring users to rewrite themselves, while maintaining new releases.

In this case, I have serious doubts about using OpenStack to deploy company services.

There is so much to understand and the need to keep pace with each project. This is awful.

And I began to realize that the customization work for Nova and Ironic is not a trivial matter, which will offset the benefits of OpenStack in open source.

But I still feel that it is very important to fully understand the details of Neutron, which is my only thought at present.

For physical switches and servers, installing and deploying servers is not too difficult, and the solution is very mature and reliable.

Automation requires a lot of tools to work with.

From my experience, the most error prone part of most basic deployment work is the automation of the network part.

You see, there are still many deficiencies in the operation system of physical switches.

The support for current automation and the interaction of API seems to be overstretched.

In fact, the poor performance of another network automation tool I used was the main reason for me to consider the use of OpenStack.

The Neutron project has a very exciting mission: it can provide scalable services that are not subject to any technology, including related libraries.

I hope so too!

But the reality is not as promised.

According to the SDN Software Defined Networking, most of the projects that work on virtual network based on virtual machine monitor (hypervisor) are not real switches.

Not only is it a serious outdated Neutron driver for switch vendors, but the support of OpenStack's latest Juno version is also limited.

In addition, Neutron uses its own imperfect IP Address Manager (IPAM), and does not have any concept of distributing its external access at all, nor does it provide written statements and authority about IP address management.

It is unacceptable to sacrifice user experience to adapt to these deficiencies of Neutron.

To make a long story short.

In the week before Christmas, we lost OpenStack and then spent three weeks developing a customized automated deployment platform.

After building their own IP management system in early December, the team was ready to build their own customized tools.

Each new project will have its own mission.

As a company, our vision is to keep forging ahead, and we feel that in the process of investigating and deploying OpenStack, most of the problems have been solved: we have built a flexible IPAM system that can provide service functions (we call it Magnum IP).

Between the facilities management platform and the physical infrastructure, we have established a user and permissions model.

Sometimes existing things are not necessarily the best, nor will they satisfy their needs.

The process of deploying packet.net with OpenStack fully illustrates this principle.

At the same time, we will also strive to release our Neutron plug-in, which is compatible with the development of OpenStack project. We are doing it now.

A week later, we finally completed the installation of the CoreOS system.