What’s the single biggest advancement Linux has brought to the industry?
It’s an interesting question, and one that in my opinion has a very simple answer: Package management—or, more specifically, the ability to install and upgrade software over the network in a seamlessly integrated fashion—along with the distributed development model package management enabled.
It used to be that operating systems were big, monolithic products, and applications were big, monolithic products you put on top of them. If you wanted to deploy, say, a web application, you sourced the middleware stack (which itself was probably several big products too), you sourced the operating system, and you (often painfully) had to integrate the two yourself (or pay a big company lots of money to do it for you).
These days, you increasingly just “apt-get install whatever“.
In this world, where does the operating system end and the application begin? The line is increasingly blurred—for when applications are deployed using an OS facility, seamlessly integrated with the OS itself, is the result an application or a feature of the operating system? In fact, in a very real way, all software looks to become part of the operating system—or, at least, this has certainly been the trend in the Linux world.
What does this shift mean to the industry? For one thing, those of us that build operating systems as monolithic products have to change—it’s what users expect, and with a componentized operating system rather than a wad of stuff, it becomes far easier to push new innovations out into the marketplace and generally evolve the OS over time. Indeed, refashioning Solaris as a “distro” is the essence of Project Indiana—and package management is the key technology that will hold it all together.
So, the next time you read about how Project Indiana aims to make Solaris more “Linux-like”, keep in mind that what we’re actually “copying” is the distro model, not Linux itself—which, after all, is a kernel, and has nothing to do with the package management and so forth the distros (you know, like Debian) built above it. This, better than anything else, highlights the opportunity for Solaris: What people really know when they say they know “Linux” is the environment that exists around the Linux kernel—the distro—all of which Solaris can deliver, and more.
We started the Blastwave project five years ago with the single intent to create and freely deliver software packages to Solaris users and we wanted to use the Debian Linux model because of its simplicity. There are currently 1695 software packages in standard SVR4 package format at Blastwave today and we are still building them such that anyone can run “pkg-get foo” and they get “foo” as well as all the dependencies. One large drawback is that we build for the Solaris 8 users as well as Solaris 9 and Solaris 10. This leads to a massive pile of duplication. It also means that the software from Blastwave is generally more up to date. Generally. It is my hope that we can continue to deliver these software packages for any OpenSolaris based distro. These packages are built based on standards, with quality assurance and bug tracking. Not perfect and not flawless but certainly with years of enthusiasm and momentum from the Solaris community. The Debian Linux approach works and it works well and it made sense to implement this model for Solaris. That is exactly what we did. Dennis Clarke, Founder Blastwave.org http://www.blastwave.org/
You may already be familiar with the “slash-package” system proposed by the infamous Dan Bernstein (qmail author and mathemetician). Paul Jarc has a lot of useful tools and documentation related to building a complete system from Dan’s ideas: http://code.dogmap.org/
The community around Paul Jarc’s or Dan Bernstein’s work doesn’t appear to be very active, but there may still be some things worth borrowing there.
There are two features in particular of the PRJ/DJB systems which I like: the filesystem is the database and anyone familiar with basic unix commands and principles automatically understands the package system. When /command/ls is a symlink to /package/base/coreutils/command/ls, it’s pretty clear that the ‘ls’ command comes from the ‘coreutils’ package. When /package/base/coreutils is a symlink to ./coreutils-5.97, it is clear that the currently active version of coreutils is 5.97. These two examples correspond exactly to dpkg -S and dpkg -l, but with the “filesystem as db” and “everything built from standard unix idioms” principles at work.
That’s why rPath has a 2 year jump on everyone. They’ve been saying this from day one and have made the conary package manager cater to this model since the beginning.
Result? They are in a better position than anyone else to support the model described above…even better than red hat.
Look to see more and more people come around posting things like Ian did here…and look for the lines between os and application continue to blur…and look for rPath to continue to have a jump on everything because they thought of it first.
Pingback: c0t0d0s0.org
Pingback: Pharao's Blog
I wholeheartedly agree that package management is a huge, huge thing.
This would be such a great opportunity for Sun to learn from DEB, learn from RPM, and then design a new, open, package management system that’s significantly better than the two (think ZFS-better) and also includes RHN-style centralized enterprise package management software that’s extensible Dag Wieers-style.
Pingback: ex tempore » Blog Archive » here we go again...
Pingback: Dalek Bell » Blog Archive » apt-get install package-management
Mikael nails it.
Here are my thoughts:
If Sun can manage to fend off the vociferous outsiders who demand instant gratification long enough to develop a packagement framework that really *is* ZFS-better, then the future of Solaris should be bright, indeed.
A majority of Free Software users are only focused App/GUI/Shell deep. They want nice applications that are easy to download, install, and run.
For example, comparisons are frequently made of Ubuntu against the rest of Distribution Universe based on the brownness of it’s default palette, default GTK theme, and the simplicity of apt-get.
Fortunately, everyone can appreciate the impact efficient package management has on the Free Software Experience.
Sure, package management and a more modern/familiar userspace on top of Solaris would be a good thing and make it more usable to the Linux crowd, but there is another thing missing.
Driver support.
Linux has it, Solaris does not.
I would love to be using ZFS right now, but Solaris does not support 3ware RAID cards, Coraid/AOE disk shelves, or even an NBD like protocol.
DarwinPorts (now MacPorts) has come along quite a ways I think and can definitely be learned from. As a long-time user of FreeBSD (since 2.1.6) and a user of all of the Linux package management systems, I see a lot of things that DarwinPorts (and portage for that matter) have gotten right.
First of all, with a source-based package system, you automatically get binary packages. So even if binary packages are going to be the defacto standard when it comes to users installing the packages, I think it’s important to have a source-based option. It makes packaging from the developer’s standpoint much easier, I think. Especially with the DarwinPorts model. As a developer, I don’t have to tell DarwinPorts what files to package up, it figures that out by what gets installed by my existing installation method (usually make install) and does the intelligent thing.
From a user perspective, it allows you to very quickly roll forward and roll back package upgrades. It keeps old versions staged in its cache and will simply unlink the new version and link the old one back into place if you wish. Similar to encap or slash-package.
Of course, the ultimate to me would be something like drag-and-drop OS X .app style management, but that would likely require a lot of third-party developer cooperation.
I’m looking forward to seeing what comes out of this.
Pingback: How package management changed everything « Bloggitation
For someone who has absolutely no knowledge about Linux, but would like to dive into Open Source computing, package management sounds like a very reassuring support feature. Enlightening piece.
For those of you who’ve yet to do so, investigate Gentoo’s ’emerge’ package management system. It’s simply amazing its versatility and robustness.
I’d only use a source based package management system. Even though, to me, package managers get in the way more often than not.
http://www.slackware.com
Pingback: FreeSoftNews » Blog Archive » How package management changed everything
Pedantic perhaps, but I don’t think it is Linux that did this, it is free software.
I was sticking plenty of apps into HP-UX all nicely packaged in the HP software distributor package format (thanks to the HP-UX porting and archiving centre) a long time before Linux was commercially viable. SUN lagged, with big admin coming along a fair while later. Whilst on the big Unices the high value software vendors would do a native package, the free software could be done on an as needed basis, or on request by the Porting and Archiving centre.
But the crucial issue is that someone who understands the platform is compiling, organizing and rearranging the software, and that requires a license that permits that (or individual agreements which can be hard to manage).
Sure some proprietary software was done the same way, SUN shipped NFS code to HP (who then failed to merge it all nicely), but then they didn’t want to the HP improvement in the SUN code base, so instead would happily introduced regressions due to merging cock-ups every other release or so.
Debian does it best (like you need telling), but that is down to the quality and variety of the packages, the tools themselves were behind the proprietary software packaging tools at the time (and probably still are).
One interesting difference though, was HP-UX shipped “patches” for years, when the Linux distros just “upgraded” packages. Which eliminated a whole patch management mess (by forcing packages to track which files they brought with them), which they persisted with for far too long. Microsoft Windows of course still goes in for patch management – ARGHHHH.
Solaris has pkgadd from its BSD roots and the FreeBSD Community has already made a pretty good pkgadd that does what Sun needs, then there’s blastwave (previous comment). I would suggest you check out dragonflybsd.org to see what the future might hold for where BSD-like operating systems are going. The advantage that Solaris has over Linux is that it has few device drivers which make it more stable and the more device drivers you add, the more likely you are to make the OS kernel LESS stable over time. Solaris could port over nooks (nooks.cs.washington.edu) since the Linux community has virtually ignored it. Obviously, building a device driver subsystem that prevents your kernel from crashing and restarts the device from previous state isn’t sexy enough, though it gets around the problem of poorly written device drivers causing trouble.
“keep in mind that what we’re actually “copying” is the distro model” –
what about the ubuntu model where ease of use for newcomers instead of developers only ;-P
Pingback: Byte Into It - Computing and new technology Byte Into It 25 Jul 2007 «
Package management is a blessing when there’s integrated QA and support. apt-get is only one half of the equasion; the Debian Bug Tracking system (and the people behind it) are the other.
Case in point: fink adds apt-get to the Mac OS, but the result is that updates from Apple and updates from fink compete/collide, and when there’s a problem, the natural result is finger-pointing rather than integrated QA.
Debian/ubuntu have scaled amazingly well, but they rely on quite a bit of trust. The MS Windows marketplace includes lots of mutually distrustful players, as does the Mac OS X marketplace. It’s not at all clear how apt-get and the debian bug tracking system (or ubuntu’s launchpad) can scale to support that sort of marketplace.
On a Mac, I’d reccomend using MacPorts, rather than Fink. MacPorts is actually run by people at apple and seems to be more in sync with the standard OSX release process.
Pingback: Byte Into It 25 Jul 2007
Absolutely spot on Ian. Regardless of whether you are managing dozens or thousands of servers, or one desktop, application management is a HUGE deal. Indeed, it can really make or break an OS. As all systems move toward well-managed package systems, application installation and removal becomes easier and less fraught with danger (e.g., bringing down a production cluster when your installation fails due to a bad compile, etc.).
While I enjoy the power and flexibility of source compiles, really, those don’t work for large installs or for your average desktop user. That’s two rather big groups.
FreeBSD, Debian, and others are getting it right. Make it easier on the sysadmin and end-user.
—
Dustin Puryear
Author, Best Practices for Managing Linux and UNIX Servers
http://www.puryear-it.com
Oh, and for when you just HAVE to do a source install, don’t forget about GNU’s Stow!
—
Dustin Puryear
Author, Best Practices for Managing Linux and UNIX Servers
http://www.puryear-it.com
G’day Ian,
I agree with your general comment in the context of GNU/Linux based systems, as such, in that the development of a good set of package management platforms indeed aided the growth and acceptance of the GNU/Linux OS as a whole – in fact it clearly provided a simple means for Operating Systems built on the Linux kernel and the GNU tool set and Unix clone file systems to move forward in leaps and bounds.
But I do need to clarify for readers of this blog post that GNU/Linux was not the first place package management was seen, nor is it the case that GNU/Linux based Operating Systems were the first to introduce, or even early adopters of package management.
A/IX, HP/UX, SunOS, Solaris (yes, even Windows) all supported package management some time before the Linux kernel was even thought of, albeit that things were not quite so kiddie-proof as they are today, package management did both exist, and essentially work.
Alas most of the “open source”, “GNU/Linux”, and now excitingly “OpenSolaris” folks are all too young to have any knowledge, history, or it seems desire to learn the history, or from whence we came. Mores the pity in my opinion.
I recall my first day at Sun, I was shown my “cubicle” I was to work in, it was literally stacked to the celing with old Sun3 systems, CPU’s, “shoe boxes” (external disk), and tape drives, and boxes of keyboards, SCSI cables, monitor cables, and power cables – and I was told “you can have this cubicle, it’s the largest.. it’s all yours.. when you’ve finished putting all these systems (old junk mostly inherited from the USA offices) on sales staff desks *chortle*”). “Welcome to Sun” and off he went (my boss).
Needless to say I learned about Sun’s package management quickly ;-)
Even before the whole “open source” movement started to take shape, SunOS “freeware” packages were shared via USENET over UUCP connections in ASCII encoded SHAR files which when “run” expanded into packages which allowed folk without compilers (yes, sadly compilers eventually were not supplied by default with Unix systems) to install tools which did not come with their Unix or Unix derivative Operating Systems as “standard software”.
So I apologise for the correction as such, indeed a small clarification I admit, but I’m sure you agree a worthwhile one.
Regards,
Dez
—
Dez Blanchfield
(ex Sun, ex HP, ex IBM, ex SGI, ex almost every Unix OEM on the planet)
http://www.Blanchfield.com.au/
http://www.WebSearch.com.au
http://www.CradleTechnologies.com/
http://www.TheStorageForum.com/