Using WebDAV and BioCoRE

WebDAV and BioCoRE

Introduction

It is possible to mount BioCoRE's BioFS filesystem from your own computer using a technology called WebDAV. This gives you many capabilities that are impossible via a standard webpage. For instance, you can copy more than one file at a time into and out of the BioFS. You can simply select the files that you want and drag them to another folder. In addition, you can often edit a file directly from the BioFS. If you and a collaborator are working on a Microsoft Word document you can edit the file directly from the BioFS, without needing to initially copy it to your local filesystem, edit it, and copy it back up.

Windows XP and 2000 and Mac OS X have WebDAV clients built-in, so that the BioCoRE WebDAV server appears as a standard network drive on your desktop. Under Linux, the davfs program will also let you mount WebDAV partitions. In addition, there are a number of file browser programs that will let you access the BioCoRE WebDAV server from a variety of clients.

Windows
Mac
Unix
Misc Notes

Windows Install

System Requirements: Windows XP or 2000 with IE6.

There are two ways to install WebDAV on Windows:

Click on "My Network Places" in the top left corner of the screen
- A website that gives pictures for this on Windows XP. Just replace the URLs in the pictures with the URL given above.
Open Internet Explorer. Click on File and then Open.
- Enter the BioCoRE WebDAV address as mentioned above. Then click on the icon "Web Folder".
- Enter your BioCoRE username and password in the box that pops up.

Opening files: double clicking will open the default application for the file, e.g. for txt/HTML files it would be IE. To edit such files they should be opened in MS word. In MS word click "open" and then "My Network Places" and then the BioCoRE WebDAV folder to navigate to the appropriate file.

For some installations of Windows you may have problems opening some types of files (txt etc.) as double clicking them may do nothing. Please update to the latest version of Windows and IE and it should solve the problem.

Mac

To connect to the the BioCoRE WebDAV server from Mac OS X, select Connect to Server... from the Go menu in the Finder. When the Connect to Server comes up, type the URL for the BioCoRE WebDAV server into the Address window. For the TCB BioCoRE server, the URL is https://biocore-s.ks.uiuc.edu/biocore/biofs/ if you are running on OS X 10.4 or later, and http://biocore.ks.uiuc.edu/biocore/biofs/ if you are running on an earlier version of OS X. For other BioCoRE servers, replace biocore.ks.uiuc.edu with the address of your BioCoRE server. Click Connect (you will probably receive a File System Security Notice on OS X 10.3 and lower. This is because HTTPS/SSL WebDAV connections were not supported, and your BioCoRE password is sent over the network as unencrypted text. If you are satisfied that your network is secure, click Continue and enter your BioCoRE username and password.) If you typed your password correctly, the WebDAV server will show up on your desktop named biofs. It is now ready for use.

OS X will store your password in the Keychain for you, so you don't have to type it every time you want to use the BioFS. You can also store references to the BioFS disk as aliases or in the Dock (on the right side) so you can easily mount it subsequently without having to retype the address.

To stop using the WebDAV server, simply eject the drive icon from the finder, or drag it to the trash.

A website that gives pictures for this on Mac OS X. Just replace the URLs in the pictures with the URL given above.

If SSL support is required, or if you're running Mac OS 8.X or 9.X, try Goliath at http://www.webdav.org/goliath.

UNIX

To use WebDAV and BioCoRE on a Unix platform, you can use one of the following mechanisms.

Using your window manager

If you are using one of the more modern window managers (such as a recent version of Gnome or KDE) there is a very high probability that you can access the BioFS directory from your window manager. This is also an excellent option if you don't have "root" on the machine that you are logged into.

Open a folder view from your window manager (usually an icon on the desktop that might be labeled something like 'Home') and type the following into the location bar at the top:

https://biocore-s.ks.uiuc.edu/biocore/biofs/

On some systems, such as KDE, you will probably need to type in:

webdavs://biocore-s.ks.uiuc.edu/biocore/biofs/

You should then be asked for your BioCoRE username and password, and then be presented with a view of the BioFS. Based on our testing, you can copy files into and out of the BioFS pretty reliably using the method, but might not have great amounts of luck opening the files directly from the BioFS within your favorite editors, etc due to lack of support within the Linux/Unix system you are using. To view or edit a file, we suggest that you first copy the file onto your own system, edit it, and then copy it back into the BioFS.

Davfs Linux Install

Davfs is a Linux file system driver that allows you to mount a WebDAV server as a local disk drive. The main davfs webpage is at:

http://dav.sf.net

and you can sometimes find newer versions at:

http://sourceforge.net/projects/dav/

Note that the "expat" library is not included in some of the Davfs distributions. It is necessary, and you can get it by downloading the cvs version of davfs. The README/install files aren't particularly clear on this. If you have any questions, let us know.

To install davfs with BioCoRE do the following:

The coda module must be loaded.
- Log in as root and run the following,
```
	# modprobe coda
```
- You will have to do this only *once* till you reboot the machine again. Alternatively you can also add it to your startup scripts so that it can be done when the system boots up.
- If you don't have the coda module, kernel "makeconfig" options to get it are under File Systems | Network Filesystems | Coda.
Installing:
- Redhat:
  If you want to enable SSL and connect to a secure BioCoRE server (and you do) you need to download the source on RH 8 and 9 and compile it yourself.
  Untar the file davfs2-x-x-x.tar.gz and build it if needed.
  For RH 8 without SSL, get davfs2-x-x-x.tar.gz (precompiled binary) from here
- Gentoo: There is a davfs2 ebuild available, but it doesn't have SSL support. There is one on bugs.gentoo.org that includes SSL, however, Download that ebuild, create the digest, and emerge it.
Download the script mount.biocore. (Note: The actual release page for these scripts is at: http://www.ks.uiuc.edu/Development/MDTools/tcb-webdav/ and you can refer to that page for additional information)
Edit mount.biocore and edit the config parameters appropriately.
The mount.biocore script is designed to reside in a binary directory somewhere in your path, and to have a symbolic link for the filename unmount.biocore pointing to it.
Run the following script > sudo mount.biocore
Run unmount.biocore to unmount biocore
davfs and 'updatedb' get along very well on your Linux system (a bit too well). Updatedb is a program that typically runs daily on your computer and catalogs the names of all of the files. This way, you can run the 'locate' command and quickly find files that you can access. Updatedb automatically follows all the davfs links. This means that every day it automatically downloads information on every single file in the BioFS that you have access to. This causes a lot of unnecessary bandwidth transfer. Anecdotally, we have observed something on the order of 5,000 hits per day, while taking in excess of an hour and a half to perform. Please configure updatedb to not follow davfs links to save both your bandwidth and ours.
To do this, you probably need to tweak the cron.daily file that your computer is using. This is often located in /etc/cron.daily/ and will require root access to modify. There will probably be a file in that directory with a name of slocate.cron or something similar. Edit the file and add your WebDAV BioFS directory to the -e option. For instance, if you have your BioFS directory mounted in /webdav/ you might have something like:
```
/usr/bin/updatedb -e "/tmp,/var/tmp,/usr/tmp,/webdav"
```
If you need any assistance with this, please email us and we can help.

Cadaver Install

Cadaver is a WebDAV client which looks a lot like an ftp client. Cadaver is available on most Unix platforms and might already be installed on your machine. The executable is normally named cadaver. Cadaver can be downloaded from here.

To connect to a WebDAV server using cadaver do the following:

The open command connects to a WebDAV server.

>./cadaver
dav:!>open http://biocore.ks.uiuc.edu/biocore/biofs
Looking up hostname... Connecting to server... connected.
Authentication required for BioCoRE on server `biocore.ks.uiuc.edu':
Username: 
Password:

It will ask for a username and password which are your BioCoRE username and password.

Use ls to list the files and directories.

dav:/biocore/biofs/> ls
Listing collection `/biocore/biofs/': (reconnecting...done) succeeded.
Coll:  All User Test Project                   0  Aug 10  2001
Coll:  MyDev                                   0  Oct  5  2001
Coll:  Private                                 0  Dec 31  1969

In the BioCoRE file system users have their own virtual roots which the projects the user has access to as sub directories.

cd changes to a subdirectory. In cadaver, directories are called Collections, or 'Coll'.

dav:/biocore/biofs/> cd MyDev
dav:/biocore/biofs/MyDev/>ls
Coll:  System                                  0  Oct  5  2001
Coll:  new                                     0  Jan 15  2002
Coll:  new1                                    0  Aug 15 08:13
       notes.txt                           17088  Aug 15 09:56
       notes2.txt                          17088  Nov  5 07:08
       test.txt                              222  Aug 15 09:56
       test.txt                              187  Oct 22 12:03

'move', 'copy' and 'rm' work the same and for other commands use 'help'.

Cadaver can be rather picky about files and directories that have nonstandard alphanumeric characters. It can be made more tolerant by using the '-t' tolerant flag, or you can put the command into your .cadaverrc file. One user suggested the following as a .cadaverrc:

set editor vim
set pager less
set tolerant
open https://biocore-s.ks.uiuc.edu/biocore/biofs/

This automatically sets the 'tolerant' flag and connects to the BioCoRE server at UIUC when you start cadaver.

Misc Notes

WebDAV has no concept of file permissions. BioCoRE's BioFS file permissions are still in effect, you just can't see them via WebDAV. If you want to change any permissions you will need to log into the web interface of the BioFS.

Most web browsers that we've seen don't want to upload files that are larger than 2 gigabytes. For that, you will probably need to use some sort of WebDAV client.