Exploring filesystems with Explore2fs

In my early dual-boot days, shuttling between my favorite Slackware Linux distribution and the Windows operating system to collaborate on projects with friends was very tiresome. Juggling files between Windows FAT filesystem and Linux's EXT2 filesystem was especially time consuming as Windows failed to recognize my Linux partitions. That's when I discovered Explore2fs (http://www.chrysocome.net/explore2fs), a little utility that enables one to easily locate and transfer files from an EXT2/3 partition into Windows. In a chat with Packt Columnist, Mayank Sharma, John Newbigin shares the status of Explore2fs and his other hacks, talks about his involvement with the CentOS project, and gives some good advice to budding FLOSS hackers.

Exploring filesystems with Explore2fs

In my early dual-boot days, shuttling between my favorite Slackware Linux distribution and the Windows operating system to collaborate on projects with friends was very tiresome. Juggling files between Windows FAT filesystem and Linux's EXT2 filesystem was especially time consuming as Windows failed to recognize my Linux partitions. That's when I discovered Explore2fs (http://www.chrysocome.net/explore2fs), a little utility that enables one to easily locate and transfer files from an EXT2/3 partition into Windows.

John Newbigin was just out of college when he wrote Explore2fs because he too was having trouble managing files between filesystems. Now a Computer Systems Officer at Swinburne University of Technology, Melbourne, Australia, he still works with the same attitude. "I don't like things that don't work properly. This drives me to fix things when I see an area for improvement. I also hate doing things more than once so I would rather produce a tool which does a job for me than to do the job twice. I often summarize this as "Work smarter, not harder", explains Newbigin.

The utility Explore2fs has come a long way. It's tried to keep pace with the developments in the file systems arena and I still recommended it to people who dual boot. In this chat, Newbigin shares the status of Explore2fs and his other hacks, talks about his involvement with the CentOS project and gives some good advice to budding FLOSS hackers.

Mayank Sharma: I've always been a big fan of your Explore2fs utility that's not only helped me but a lot of my peers as well. How did Explore2fs come about?

John Newbigin: In mid 1997 I started an industry based work placement at my University (Swinburne University of Technology). I was sharing an office with another guy from my course. We did not have much guidance as to what we were meant to be doing, so we ended up basically doing whatever we wanted. He got me hooked on Linux. At home, I had a dual boot Windows and Linux computer. Being interested in networking, I would use Linux to dial up the Internet and amongst other things, download MP3 files. When I booted back into Windows, I wanted to play the MP3 files but they were on the Linux file system, and that is when Explore2fs was started.
 

MS: Can it still only read EXT partitions? How has the utility grown over the years? Are you still working on it? What major improvements/features can we expect?

JN: Explore2fs can read ext2 and ext3 file systems. When it was first written, ext3 did not exist but it is designed so that as long as the file system is cleanly dismounted, it can be treated the same as ext2.

In the beginning, Explore2fs had read and write support but there were some bugs in the write support which were causing problems so in the end it was removed. Explore2fs is written entirely in Delphi. All the header files were manually converted to Delphi and I re-implemented all the required functions. This was great for learning the file system but not so great for maintainability. As new features were added to the file system, I had to learn what the new feature was and implement support for it. This became harder to do once I started working full time and had less time to spend on Explore2fs.

There have also been a lot of new disk technologies that came about in the past few years. A major one being LVM2 which is now common even on home Linux installs. These posed new problems for Explore2fs as it tried to remain compatible. All the signs were pointing to a new, well designed, and modular version of Explore2fs.

This year I started a new project called Virtual Volumes. It uses a modular design to support multiple storage technologies like Partitions, Volumes, Software RAID, LVM2, and loopback mounted files. There are plans to add encryption and network protocols like iSCSI as well.
 
There is a new Virtual File System (VFS), which supports many file systems and allows them to be mounted in a unified tree. Some file systems require the disk I/O layer and others to use an API or network protocol. Current file systems include ext2/ext3, win32 API (FAT and NTFS), sftp, and a memory-based temporary file system. Ext2 support now uses libext2fs and has read/write support.

There are plans to add ISO9660, FAT, HFS+, and ReiserFS.
 

MS: Most of your projects seem to be for Windows users. Is that deliberate or accidental?

JN: Before I got into Linux, I mostly programmed in Delphi 2. I had done a lot of DOS programming in BASIC, Pascal, C, and C++ before that but none of those languages are good for building GUIs. When I started to get into Linux, my tools then grew out of necessity. I wanted a Windows GUI to help perform some Linux related task. For example, Windows does not come with a tool to put a Linux boot disk image onto a boot disk. Under DOS and Win95 you can use the DOS rawrite program, but that did not work under NT. This was the reason I wrote RawWrite for Windows.

Explore2fs was similar in that there was a DOS program to read ext2 but nothing good for Windows. It should have been easier in Windows because of the better disk IO subsystem. There were a few tools which did one thing or another but no fully integrated program. It was also a good learning exercise in Windows programming, file systems, and testing a complex program.

 

MS: Tell us a little about your other hacks/projects.

JN: I have created a lot of small tools (http://www.chrysocome.net/projects); mostly Linux based which all make my life easier. I have done a lot of SMB (Windows) based authentication tools to allow Linux to exist in a Windows domain environment. SMB Authentication for pppd and php allow us to provide services to Windows users and have them remain unaware that it is a Linux-based service.

I also implemented CIFS extensions for UNIX in the Linux kernel, smbfs driver, which allows one Linux machine to mount a directory from another Linux machine and have UNIX file information available, making smbfs an alternative to NFS.

One tool I wrote and use regularly is ndu. This is a simple tool which reads your bind zone file and keeps your Reverse DNS correct. mysqldict is another tool which can be used to keep your mysql database tables correct and is invaluable if you are doing some software development and keep both a test database and a live database.
 

MS: You are also involved with the CentOS distribution. How did that happen?

JN: I have been using Red Hat Linux since version 5.2. I first introduced Red Hat Linux to my department at version 6.2. We then upgraded to 7.2. The .2 releases were always the best. We planned to upgrade to 8.2 but that never happened. Red Hat changed their development plans and we, like many departments in the world were left with a choice of purchasing Red Hat Enterprise Linux (RHEL) or switching to a different distribution. Red Hat was unwilling to offer educational pricing to us which ruled out RHEL, but we were reluctant to switch to something else. I toyed with the idea of using a RHEL rebuild but no one had produced one which I found was good enough and most people were focusing on the new RHEL version 3.

It was then that I decided to build my own RHEL2.1AS. One group who had been playing with a rebuild already was The cAos Foundation. They wanted to use it as a platform for building their own distro. I teamed up with them to produce CentOS-2. They had already produced the replacement artwork and had settled on yum for delivering updates. Soon after, CentOS split from cAos and became a stand alone project.
 

MS: What are the various CentOS development teams? What does your team, CentOS-2, do?

JN: Although anyone is free to contribute, there are not many CentOS developers.

I am the only person working on CentOS-2. It is at the stage where no new CD images are produced, just updates which can be downloaded over the Internet. Building these updates is scripted so there is not much for me to do but run a few scripts when a new update is released.

CentOS-4 is where most of the action happens these days. I am planning to package my CentOS-2 build environment into a virtual machine, which would allow other developers to have access to it. This will form a bit of a contingency for if I am unable to respond to updates in a timely manner.
 

MS: What would be your advice to budding FLOSS hackers?

JN: Anyone can be a FLOSS hacker but cracking into a development community can be difficult. Here is some vague advice.

Do lots of practice. Many projects these days produce stable and mature products. The developers who work on these projects may have been programming for 20 years and working on the specific project for 10 years. A newcomer can find it hard to compete with such experience. You need to make sure that before you put your hands up, your skills are worthy of attention. This means that if you can't code up a binary search or validate your XML, you still have homework to do. Get a book on computer science or programming or whatever topic you are interested in and learn the content. Even if you want to help do graphics for a
web-based application, you need to make sure that your contribution will help the project, not just distract the developers from their tasks.

If you don't know where to start, there are lots of resources on the Internet which cover all sorts of topics in all sorts of depths. Find a newsgroup or forum which helps teams whatever it is that you are interested in.

Get your hands dirty. The next step is to look under the hood of a project and see how it works. Many FLOSS developers are not good at documentation so there might not be instructions on how something works. There will still be a wealth of information in the source code and in mailing list archives. Most FLOSS developers don't like answering the same question twice so give Google a work out. Once you are comfortable with making changes to the code, recompiling, etc., then you are ready to throw your hat into the ring.

Have something to prove your worth. This is the final step and stems from the previous point. If you jump on the list and say, "I want to do blah, blah, blah", you might not get as good a result than if you say "I have done a prototype for blah, blah, blah, what do you think of it?".

Digg!
del.icio.usSave to del.icio.us

Mayank Sharma is a freelance writer from New Delhi, India. He is blown away by the power of Free and Open Source Software and its usefulness to developing nations.

Check out his blog at http://www.geekybodhi.net/

 

   
Code Download and Errata
Packt Anytime, Anywhere
Register Books
Print Upgrades
eBook Downloads
Video Support
Contact Us
Awards Voting Nominations Previous Winners
Judges Open Source CMS Hall Of Fame CMS Most Promising Open Source Project Open Source E-Commerce Applications Open Source JavaScript Library Open Source Graphics Software
Resources
Open Source CMS Hall Of Fame CMS Most Promising Open Source Project Open Source E-Commerce Applications Open Source JavaScript Library Open Source Graphics Software