PSA: Is your Ubuntu Server IaaS Guest Image Authentic?

The amount of uptake seen with Ubuntu Server over the past year has been extremely rewarding and simply amazing.  Infrastructure as a Service (IaaS), a.k.a. Public Cloud, providers are popping up left and right, all wanting to provide Ubuntu Server…all helping to further cement Ubuntu Server’s position as the OS for the cloud.

With that said, I’ve started to become concerned about the way in which some of these IaaS providers distribute Ubuntu.  Ubuntu developers create, publish, and regularly update images on Amazon Web Services and Microsoft Azure.  Canonical hosts and maintains internal archive mirrors in these clouds to provide a low-latency, low-cost update mechanism to users.  Finally, Canonical engineers purposely designed in a pluggable cloud provider API approach to Ubuntu’s service orchestration application, Juju, to lower the operational barriers that often place limitations on cross-cloud workload and service migrations.  We do all this to help ensure cross-platform consistency for Ubuntu Server users, i.e. workloads and applications ran on Ubuntu Server behave in the same manner on bare metal machines and across IaaS providers.

Some IaaS providers and users have decided to produce and host their own Ubuntu Server images without the involvement of the Ubuntu Project or Canonical.  I won’t go into the legal aspects of this, because I’m no lawyer.  However, I believe there is a real risk to users when these images are modified in some way, but still presented as “official” Ubuntu Server images.  Whether the changes are minor, like redirecting fixes and security updates to internal unofficial mirrors, or major, like making changes to OS and/or applications provided in the images themselves, labeling the images as “official”Ubuntu Server is a misrepresentation of the project and the product.  There is a real and legitimate risk of users losing out on the cross-platform assurance that the Ubuntu project and Canonical work so hard to provide due to the images having untested code or simply being out of sync on fixes and updates.  Furthermore, there’s no guarantee that bug fixes made to these modified images will ever make it into the official distro, thus creating a further fork between expected behavior across both bare metal and cloud platforms.  All of this has the potential to lead to poor user experience that’s very damaging to the reputation of Ubuntu the project and product, not to mention Canonical as it’s sponsor.

We ,within the Ubuntu Server team, work extremely hard to ensure our community can depend on having the same user experience and application execution results across all supported platforms, bare metal or cloud.  So…if you are a IaaS provider, and you elect to produce and distribute modified Ubuntu Server images, please…please ensure your users are aware of this by labeling them as customized derivatives.  Let them know that by using these modified images they potentially run the risk of being delayed in getting bug fixes and security updates…and that differences in OS and application behavior from your changes can lead to higher levels of complexity if/when they have a need to move workloads and services to/from other official Ubuntu Server deployments.

Thanks…we now return you to your regularly scheduled program. 😉

Advertisements

About Robbie

I live in the awesome city of Austin, TX and work for Canonical, sponsors of the best damn operating system in the world...Ubuntu.

Posted on June 26, 2012, in Ubuntu and tagged . Bookmark the permalink. 17 Comments.

  1. Can you describe a process or mechanism by which a customer can attempt to self-verify if their guest instance is official or not?

    it’s one thing to request providers to notify users of customization..but if users who care can’t verify whether they are using an official guest image I don’t see the point. If there is no established verification procudure that an end-user can use then “official/genuine” becomes difficult to establish.

    -jef

    • The easiest way is to simply send email to ubuntu-cloud@lists.ubuntu.com and ask. With that said, I realize this approach can be a publicity issue for the user, and thus quite sub-optimal. Over this next cycle we will be looking into alternative, more user friendly/less public methods for users to verify their images.

      • What you just gave me was a way to see if a vendor has registered with Canonical as running an official image. Different answer to a different question than I asked. Why on eart would asking a list be a verification process?

        Let me restate for clarity.

        I find myself running a guest image on a host provider. How do I self-verify by just examining the image itself that it is genuine? Surely there is a way to fingerprint an official guest image and then to compare that fingerprint with a published fingerprint. Some sort of complicated chksumming procedure. So sort of signature verification? Something…that is actually a technical “verfication” and not a 3rd party “registration”

        -jef

      • If they are using stock iso images or our created images hosted at http://cloud-images.ubuntu.com/precise/current/, I’m assuming the chksums should match if they haven’t modified the images.

      • Uhm…

        It is not my understanding that you use those checksums to verify that a providers guest image is official. That seems to imply that you can download the guest image locally from the provide and run the checksum.

        Do the providers you are most concerned with let you do that… download the guest image outside of their infrastructure so you can independently verify it?

        You purchase/initiate a specific branded guest OS instance subscription from a provider. You login into the guest instance. As a customer, how do you verify its the advertised guest system is the one that was advertised and indeed has not been customized to any extent? I don’t understand how the image checksums cane be used for that. I can se how a provider uses the checksums…but not the customer who is looking to verify the provider is living up to advertised claims.

        Based on my more intimate knowledge of other packaging systems.. I sort of naively assumed that the Ubuntu packaging system would have a packaging signing system and filesystem verification which could be used to used to provide a fingerprint/verificatoin system for images. First you check that installed packages are all official and signed with an official Ubuntu key to rule out package level customization. Then once that is established you verify that all package payloads installed on the filesystem have expected chksums to match what the packaging system expects to rule out direct filesystem customization. Is that sort of verification process not possible on Ubuntu cloud images for some reason?

        -jef

      • How a provider gives access to an image is up to the provider. If you can’t get access to the image file, then yes, you must then verify the signature of every package installed after initial bring up, and after each and every update applied.

  2. Jo-Erlend Schinstad

    I’m not entirely convinced that it’s automatically legal for cloud services to create custom images and label them as Ubuntu Server? I mean there is a trademark issue as well. In any case, I think it would be quite rude to do so. A slight name change to reflect the fact that it’s customized seems obvious.

  3. Can I ask a dumb question? Isn’t Ubuntu Server open source? I mean can’t I just host my own images and then spin up as many servers and hosting as I need to do?
    Don’t mean to sound like an idiot, but that’s what I thought.

    • Yes you can. The issue arises if you start changing the software contained in the distro and still present it to users as “Ubuntu Server”. For example, you can remove the init system and replace it with your own, for your own use and the use of others. However, you can’t then present it to users as “Ubuntu Server”, b/c it’s not….it’s a customized version of Ubuntu Server.

      • Thanks for the quick response, wasn’t trying to be a smart ass or anything just trying to better understand the complaint or the issue. So the concern is if I just change the software distributed not the fact that I’m not using an official Canonical provided image in my own cloud?

      • That’s tricky, because if you start to assert your trademark too much, you may lose goodwill ( remeber the story of UbuntuOne ), despites being right. On the other hand, if you don’t, the project may indeed be seen as lower quality than what you would. From my own experience at Mandriva, there is no good way to win that, so you have to choose between the 2. And yes, sooner or later, people will start to try to add values by modifying the distribution, and that’s where problem will start ( cause of course, if someone is able to modify the distribution, you want to keep them as contributor, but if you start to charge money for that, they may not have it and may prefer to use Debian or Centos ).

      • Michael,

        If Canonical does what they have historically done, then sadly,
        yes this is a warning shot across the bow ahead of C&D letters for
        trademark enforcement. We\’ve seen them take a very aggressive stance with projects like eeebuntu which is now ”easy peasy” to protect the *buntu marks in the past…especially once a project started to generate revenue (selling pre-installed media for example).

        The cloud providers are generating significant revenue from the
        Ubuntu marks. I fully expect Canonical to continue to use their
        trademarks to go after revenue generation that abuse the marks.

        Beyond that…If those providers aren’t actually providing
        ”Ubuntu” as advertised..then that is a problem for
        everyone..including the vendor though they might not know it.

        The question that ”Ubuntu: the project” needs to try to figure
        out is how to appropriately make space for multi-vendor value-add
        other than Canonical. There is a need to layout some good faith best
        practices for Cloud providers who want to bundle in value-add on top
        of Ubuntu server. Best practices for how to bundle add
        ons/customizations into pre-cooked cloud images so users can disable
        the value-add easily and get back to a stock U server experience if
        they want. Best practices for how to advertise that its a Vendor specific value-added pre-cooked image based on U server.

        These are all solvable problems with pragmatic thoughtful policy decisions at the Ubuntu project level. Once “Ubuntu: the project” recognizes that making it possible for providers to compete
        with each other at the value-add level without fighting the project
        wide branding policies is actually in the best interest of the project.

        If ”Ubuntu: the project” continues to only make room for
        Canonical’s value-add interests in the trademark policy and in baked-in functionality, it only hurts the project long term. These providers are the _profit_ generators in the cloud..they have the revenue streams to generate self-sustaining multi-vendor investment into Ubuntu as a preferred guest OS long term. Give them an equitable level playing field to use the Ubuntu marks, and encourage them to invest directly in project development. Don\’t beat them over the head with trademark policy that is written to provide preferential treatment to a single business entity that is competing for mangement value-add.

        -jef

  4. I wonder, there is a big difference between an image with a different init system and an image with some interesting tool that goes bleep when the toast is ready. Surely while one would be a trademark problem, the other would merely be seasoning on what would otherwise be a bland Ubuntu Server image?

    Also, some kind of automated scripts for checking all package signatures and installed files would be useful. Something that can pull back a report showing how it’s different.

    • Martin,
      I think the issue here is just making sure its clear what you are getting before you lay money down. For your toast alarm example.
      A designation like Ubuntu Server ++ and the provider is required to list out what the ++ means in some fashion.

  5. Reblogged this on More Mind Spew-age from Harold Spencer Jr. and commented:
    Good insight as to the hard work Ubuntu puts into their UEC images. A highly recommended read…..

  6. For anyone interested in the details of Ubuntu’s trademark and copyright policy, please see -> http://www.ubuntu.com/aboutus/trademarkpolicy

  1. Pingback: Ubuntu developers: Robbie Williamson: PSA: Is your Ubuntu Server IaaS Guest Image Authentic? | Linux-Support.com

Leave a Reply

Fill in your details below or click an icon to log in:

WordPress.com Logo

You are commenting using your WordPress.com account. Log Out / Change )

Twitter picture

You are commenting using your Twitter account. Log Out / Change )

Facebook photo

You are commenting using your Facebook account. Log Out / Change )

Google+ photo

You are commenting using your Google+ account. Log Out / Change )

Connecting to %s

%d bloggers like this: