Lecture 40

Basic Abstraction: FILE (regular file)

• array of bytes
• has low-level name (i.e., a number)
• can create/read/write/grow/delete
• persistent

Basic Abstraction: DIRECTORY (special type of file)

• array of records
  • Each entry in a directory refers to either files or other dirs
• maps human-understandable file/dir name to low-level file/dir name
• can create/delete/read (but not write to, directly)

FILE DESCRIPTOR

• all open files are tracked in a table of open files (not shared with other processes, in general)
• file descriptor is an integer which each file uses to point to a given open file in that table
• tracked per open file: CURRENT OFFSET
  • incremented after a read(), write()

Example Directory Tree

File creation

#include <stdio.h>
#include <sys/types.h>
#include <sys/stat.h>
#include <fcntl.h>
#include <assert.h>

int
main(int argc, char *argv[])
{
    int rc = creat("file1", S_IRWXU); //open("file1",O_CREAT,S_IRWXU);
    assert(rc >= 0);
    return 0;
}

• creat system call is open system call with O_CREAT | O WRONLY | O TRUNC flag
• open/create return a file descriptor

int fd = open(``foo'', O_CREAT | O_WRONLY | O_TRUNC);

Per-process management of FDs

struct proc {
  ...
  struct file *ofile[NOFILE]; // Open files
... 
};

OFT

• Each process maintains an array of fds, each of which refers to an entry in the system-wide open file table.
  • Each entry in OFT tracks the underlying file, the current offset, readable/writeable, etc

struct file { // simple version
  int type;   // e.g., FILE, PIPE
  int ref_cnt; 
  int mode;   // e.g., READABLE, WRITEABLE
  int offset; // current offset

  struct inode *inode;
  // or pointer to pipe, etc.
}

• Usually, one-to-one mapping of per-process FD -> struct file
• However, in some cases, can have multiple FDs -> one struct file, e.g., fork and dup

Reading and Writing Files

prompt> echo hello > foo
prompt> cat foo
hello
prompt>

strace for tracing system calls made

prompt> strace cat foo
…
open(“foo”, O_RDONLY|O_LARGEFILE)   = 3
read(3, “hello\n”, 4096)    = 6
write(1, “hello\n”, 6)      = 6 
hello
read(3, “”, 4096)           = 0 
close(3)                = 0
…
prompt>

• FD 0,1,2 for stdin, stdout, stderr. So, foo gets FD 3
• read syscall takes a FD and a buffer ptr, where the result will be placed
• write syscall takes a FD and a buffer ptr. Here, the write is to stdout

Non-sequential read/write

• An open file has a current offset .

Updating offset

Examples of seeking

Opens a \(300\) byte file and read it again and again..

Open the same file twice and issue a read to each of them

fds 3 and 4 refer to Open file table (OFT) entries 10 and 11

use lseek() to reposition the current offset before reading

fsync()

• FS stores writes in memory, before writing to disk
• If an application wants immediate write to disk, use fsync()

Example

int fd = open"foo", O_CREAT | O_WRONLY | O_TRUNC;
assert(fd > -1);
int rc = write(fd, buffer, size);
assert(rc == size);
rc = fsync(fd);
assert(rc == 0);

Renaming Files

prompt> mv foo bar  // mv uses the system call rename

Vim may save files like this

int fint fd = open("foo.txt.tmp", O_WRONLY|O_CREAT|O_TRUNC);
write(fd, buffer, size); // write out new version of file
fsync(fd);
close(fd);
rename("foo.txt.tmp", "foo.txt");

What info should a file system store about a file?

• all of this information is stored in a per-file structure called an “inode”
• KEY: low-level name of file is thus the “inode number”
  • inode is short for index node, because original unix FS stored all inodes in an array on disk, indexed by this inode number

Other file info

• Size and blocks
• Ownership and permissions
• Access times
• Pointers to blocks

Get file info using stat

struct stat {
dev_t st_dev;   /* ID of device containing file */
ino_t st_ino;   /* inode number */
mode_t st_mode;     /* protection */
nlink_t st_nlink;   /* number of hard links */
uid_t st_uid;   /* user ID of owner */
gid_t st_gid;   /* group ID of owner */
dev_t st_rdev;  /* device ID if special file */
off_t st_size;  /* total size, in bytes */
blksize_t st_blksize; /* blocksize for filesystem I/O */
blkcnt_t st_blocks;     /* number of blocks allocated */
time_t st_atime;    /* time of last access */
time_t st_mtime;    /* time of last modification */
time_t st_ctime;    /* time of last status change */
};

Usage

prompt> echo hello > file
prompt> stat file
File: ‘file’
Size: 6 Blocks: 8 IO Block: 4096 regular file
Device: 811h/2065d Inode: 67158084 Links: 1
Access: 0640/-rw-r----- Uid: 30686/ root Gid: 30686/ remzi
Access: 2011-05-03 15:50:20.157594748 -0500
Modify: 2011-05-03 15:50:20.157594748 -0500
Change: 2011-05-03 15:50:20.157594748 -0500

Removing Files

prompt> strace rm foo
…
unlink“foo”     = 0 // return 0 upon success
…
prompt>

Why unlink and not remove/delete?

Hard Links

prompt> echo hello > file
prompt> cat file
hello
prompt> ln file file2  // create a hard link, link file to file2
prompt> cat file2
hello

• link() syscall creates another name in the directory you are creating the link to, and refers it to the same inode number of the original file; file isn’t copied

prompt> ls -i file file2
67158084 file  /* inode value is 67158084 */
67158084 file2 /* inode value is 67158084 */
prompt>

• Thus, to remove a file, we call unlink and decrement reference count

prompt> rm file
removed ‘file’
prompt> cat file2       // Still access the file
hello

prompt> echo hello > file           /* create file*/
prompt> stat file
... Inode: 67158084 Links: 1 ...    /* Link count is 1 */
prompt> ln file file2               /* hard link file2 */
prompt> stat file
... Inode: 67158084 Links: 2 ...    /* Link count is 2 */
prompt> stat file2
... Inode: 67158084 Links: 2 ...    /* Link count is 2 */
prompt> ln file2 file3              /* hard link file3 */
prompt> stat file
... Inode: 67158084 Links: 3 ...    /* Link count is 3 */
prompt> rm file                     /* remove file */
prompt> stat file2
... Inode: 67158084 Links: 2 ...    /* Link count is 2 */
prompt> rm file2                    /* remove file2 */
prompt> stat file3
... Inode: 67158084 Links: 1 ...    /* Link count is 1 */
prompt> rm file3

Symbolic Links (Soft Link)

• Create a symbolic link: ln -s

prompt> echo hello > file
prompt> ln –s file file2  /* option –s : create a symbolic link, */
prompt> cat file2
hello

A symbolic link holds the pathname of the linked-to file as the data in a link file

prompt> stat file
... regular file ...
prompt> stat file2
... symbolic link ...   // Actually a file it self of a different type
prompt> ls -al
drwxr-x---  2 remzi remzi   29 May 3 19:10 ./
drwxr-x--- 27 remzi remzi 4096 May 3 15:14 ../          // directory
-rw-r-----  1 remzi remzi    6 May 3 19:10 file         // regular file
lrwxrwxrwx  1 remzi remzi    4 May 3 19:10 file2 -> file    // symbolic link

link to a longer pathname leads to a bigger link file

prompt> echo hello > alongerfilename
prompt> ln -s alongerfilename file3
prompt> ls -al alongerfilename file3
-rw-r----- 1 remzi remzi  6 May 3 19:17 alongerfilename
lrwxrwxrwx 1 remzi remzi 15 May 3 19:17 file3 -> alongerfilename

Dangling reference

prompt> echo hello > file
prompt> ln -s file file2
prompt> cat file2
hello
prompt> rm file     // remove the original file
prompt> cat file2
cat: file2: No such file or directory

Directories

• a directory is just a SPECIAL TYPE of FILE
• can create/read/delete (but NOT write to, directly)
• stores mapping of human-readable name to low-level name (inode number)
• When a directory is created using mkdir, it is empty, but has “.” and “..”

prompt> strace mkdir foo
…
mkdir“foo”, 0777        = 0
prompt>
prompt> ls –a
./  ../
prompt> ls -al
total 8
drwxr-x---  2 remzi remzi    6 Apr 30 16:17 ./
drwxr-x--- 26 remzi remzi 4096 Apr 30 16:17 ../

Reading Directories

int main(int argc, char *argv[]) {
  DIR *dp = opendir(".");       // open current directory
  assert(dp != NULL);
  struct dirent *d;
  while ((d = readdir(dp)) != NULL) {
      printf("%lu %s\n", (unsigned long) d->d_ino, d->d_name);
  }
  closedir(dp);
  return 0;
}

Directory entry

struct dirent {
  char      d_name[256];    /* filename */
  ino_t     d_ino;      /* inode number */
  off_t     d_off;      /* offset to the next direct */
  unsigned short    d_reclen;       /* length of this record */
  unsigned char d_type;     /* type of file */
}

Delete directories using rmdir (need the dir to be empty)

Making and Mounting a File System

• Each file system is stored in a “VOLUME”
• looks like a disk (but could be part of a disk, a partition)
• but all stitched together into SINGLE TREE of all file systems
• mkfs tool : Make a file system using a device and fstype

prompt> mount –t ext3 /dev/sda1 /home/users
prompt> ls /home/users
a b

• mount: shows what is mounted

/dev/sda1 on / type ext3 rw
proc on /proc type proc rw
sysfs on /sys type sysfs rw
/dev/sda5 on /tmp type ext3 rw
/dev/sda7 on /var/vice/cache type ext3 rw
tmpfs on /dev/shm type tmpfs rw
AFS on /afs type afs rw

• ext3: A standard disk-based file system
• proc: A file system for accessing information about current processes
• tmpfs: A file system just for temporary files
• AFS: A distributed file system