4 Replies Latest reply: Feb 29, 2012 8:22 PM by darren hartford RSS

compressedfs/SquashFS & unionFS with boxgrinder

darren hartford Expert

Hey all,

This is likely a newbie question in the virtualization/boxgrinder environment, but is it feasible (and good idea or cons to be aware of) for having the base OS stored on compressedFS or SquashFS to further reduce OS diskspace on these virtual environments?

 

EDIT: context is as server vm's, such as amazon, xen, kvm running tomcat or jboss app servers

 

Thanks for any feedback!

 

-D

  • 1. Re: compressedfs/SquashFS & unionFS with boxgrinder
    msavy Novice

    Hi Darren,

     

    Could qcow2[1] be what you're looking for?

    $ boxgrinder-build my-appliance.appl --os-config format:qcow2

    [1] http://people.gnome.org/~markmc/qcow-image-format.html point 2

  • 2. Re: compressedfs/SquashFS & unionFS with boxgrinder
    darren hartford Expert

    I read up a little on qcow, but it sounded more like the 'image container' than the underlying operating system filesystem.  Again, ignorance on my part is what I'm trying correct,  would you have a 'qcow' container image file that has XYZ-linux with ext3 (or btrfs) filesystem normally...or am I misunderstanding that?

     

    If that's the case, great, that's another option (as well as another whole slew of variables to review ;-) .

     

    I've seen some of the xen images use compressed/squashFS as they are unlikely to change and only provide unionfs for those few times that it needs to, while other parts of the system run a more normal filesystem(ext3).  As to if this is always a benefit, or if there are downsides, I'm unsure nor sure if boxgrinder could support something like it anyway (if it was a good idea). 

     

    (caveat: I'm not a diskspace tight-wad, but there are more than you would think especially when you talk iscsi/san/tier-1 style storage versus just local disk access)

  • 3. Re: compressedfs/SquashFS & unionFS with boxgrinder
    msavy Novice

    qcow2 uses sparse disk techniques, so only the 'actually' occupied space is reserved on the real disk.

     

    e.g.

    ├── [1.2G]  boxgrinder-meta-sda.qcow2

    ├── [ 10G]  boxgrinder-meta-sda.raw

     

    You treat it just like you would a raw disk, it is (effectively) transparent to the internally running system. So your appliance can have whatever filesystem you desire.

     

    If you are just referring to the system internals, at present we only officially support ext3 or 4 in the 'partitions' section of the appliance definition. However, there isn't anything to stop ingenious fiddling with scripts in the post section to convert partition(s) to whatever format you fancy.

  • 4. Re: compressedfs/SquashFS & unionFS with boxgrinder
    darren hartford Expert

    msavy wrote:

    ...

    If you are just referring to the system internals, at present we only officially support ext3 or 4 in the 'partitions' section of the appliance definition. However, there isn't anything to stop ingenious fiddling with scripts in the post section to convert partition(s) to whatever format you fancy.

     

    Thanks, that does clarify something I wasn't sure about...all the o.s. options obviously supported many more filesystems, but the current appliance definition partitions (ext3 or ext4 only atm) in boxgrinder is where I wasn't sure about.  The new information about qcow2 (for me) is also very helpful, but as always need to test out scenarios (phoronix baselining for example) to understand the impacts versus the gains.

     

    Thanks, and if anyone has more information or has done similar benchmarking (compression rates and decompression times (i.e. startup time for example on compressed base o.s. filesystems)) please share.