Milestone 1‎ > ‎

Detailed specification

Detailed assignment description

Storage server functionality

The storage server is organized into tables, and each table contains a collection of records, where each record is identified by a unique key. For this version of the storage server, each record is a simple string with a maximum number of characters. 

For example, suppose you wish to store your marks in a table called marks, using the course code as the key to identify each record. The marks table may look like this. 

 Key  Value
 ECE100  78
 ECE143          92
 ECE297  87

The storage server supports the following functionality:

  • Retrieve an existing record. (Use the storage_get() function.) 
  • Insert a new record. (Use the storage_set() function with a key that does not already exist in the table.) 
  • Update an existing record. (Use the storage_set() function with a key that already exists in the table.) 
  • Delete a record. (Use the storage_set() function with a NULL value.) 

The detailed specification of the storage_get() and storage_set() functions is described below in Section 1.3. 

Storage server skeleton code functionality


The skeleton code contains the basic framework for the storage server and the client that can communicate with the server (see figure above). In its current state, the skeleton code contains the application programming interface (API) for the functionalities provided by the storage server. The API is the interface provided by the storage server. From Milestone 2 onward, you will implement this functionality according to our specification. For this milestone, Milestone 1, you will implement a command-line shell to allow users to interact, through the client, with the storage server. The shell will allow the client to call the appropriate functions from the client library, which will send the command over the network to the server.

The skeleton code is divided into three components: the client, the client library, and the server:
  • Client: The client contains the shell, which is used to receive commands from the user (through the command-line). The client must parse the input properly and call the appropriate functions from the client library. In the current state of the skeleton code, the client is not interactive and instead calls a fixed set of commands every time.
  • Client library: The client library contains the logic necessary to communicate with the server. In its current state, the client library is able to establish a socket connection with the server and send commands over the network to the server. However, it lacks the ability to understand replies from the server.
  • Server: The server contains the core of the code, as it is in charge of the storage and querying of data. The server must be able to handle commands sent from the client (through the client library), execute the appropriate command and send back the correct reply. In its current state, the skeleton server is only able to accept incoming socket connections, receive commands and log them. 
Section 2 describes how to obtain the skeleton code and run it on the lab machines.

Milestone 1 storage server specification

You must implement a command-line shell inside the client. The shell accepts input from the user and translates the input into client library calls. The calls will be forwarded to the server using the skeleton's client library implementation. You do not have to worry about the actual communication, which is already implemented in the skeleton code. It may however be instructive to read the code.

You must then replace and add logging calls in the code with a new logger() function found in the utils.c file. This logger() should be able to output logging messages to the appropriate output stream (either stdout or a file), depending on the constants provided to the client or server.

Command-line shell

Although the storage server will not be functional until Milestone 2, it is beneficial to implement a command-line shell first to ease implementation and debugging of the project as a whole. An interactive shell will allow you to easily enter commands to your server and pinpoint corner cases where your code is not working, which will speed up the debugging process for subsequent milestones immensely. It is therefore imperative that the command-line shell is implemented first. Furthermore, the command-line shell is required for the midterm, where you will be required to demonstrate the functionality of your storage server. You will be asked to submit commands to your server through your client shell.

The shell must print a menu containing the available options to the user:
  1. Connect: Allows the user to connect to the server at the specified hostname and port.
  2. Authenticate: Allows the user to authenticate using the specified username and password.
  3. Get: Allows the user to retrieve the value of the specified key from a table.
  4. Set: Allows the user to set the value of a specified key from a table.
  5. Disconnect: Terminate the current connection.
  6. Exit the client
Items in bold correspond to values that must be input by the user. The shell must include a prompt to allow the user to input such values. Your shell must then take those values and make the appropriate function call to the functions provided in the client library.

The functions in the client library are specified in the below table. Note, in this milestone, you are not asked to implement this functionality. You are only asked to pass the user input from the shell on to the server via the functions provided in the client library.

Function Parameters Description
storage_connect

hostname: The IP address or hostname of the server.
port: The TCP port of the server.

Establish a connection to the server running at hostname on port port.
storage_auth

username: Username to access the storage server.
passwd: Password in plain text form.
conn: A connection to the server.

Authenticate the client's connection to the server.
storage_get

table: A table in the database.
key: A key in the table.
record: A pointer to a record structure.
conn: A connection to the server.

The record with the specified key in the specified table is retrieved from the server using the specified connection. If the key is found, the record structure is populated with the details of the corresponding record. Otherwise, the record structure is not modified.
storage_set

table: A table in the database.
key: A key in the table.
record: A pointer to a record structure.
connection: A connection to the server.

The key and record are stored in the table of the database using the connection. If the key already exists in the table, the corresponding record is updated with the one specified here. If the key exists in the table and the record is NULL, the key/value pair are deleted from the table.
storage_disconnect

connection: A pointer to the connection structure returned in an earlier call to storage_connect.

Close the connection to the server.

* Note, you can ignore storage_query for now, it is only used in Milestone 3 onward.

The skeleton code already contains a sample function call for each of the available options detailed above. Analyse the existing code carefully to understand how the shell will make those calls on its own. Your shell does not have to perform input validation and can therefore crash if an unexpected value is entered: It is however strongly recommended that you add input validation to your shell for your own convenience and for the midterm exam, where a reliable shell will save you time from mistakes. In general, it is good coding practice to implement input validation in your code.

Here is a sample shell dialogue. Note, you do not have to use the exact same wording as in this example.


> ---------------------
> 1) Connect
> 2) Authenticate
> 3) Get
> 4) Set
> 5) Disconnect
> 6) Exit
---------------------
> Please enter your selection: 1
> Please input the hostname: localhost
> Please input the port: 1111
> Connecting to localhost:1111 ...

Also note that functions implemented in the client library will already output some feedback (e.g., error messages). You do not have to modify or delete those statements, but you feel free to output more information through the shell.

Logging

You will also implement logging for this milestone. Logging, along with the shell, is an essential tool for debugging. By logging the appropriate data at the appropriate locations in your code, you can retrace the execution of your program and discern parts of your code which are faulty. Quickly locating the source of bugs will ease the troubleshooting process later on.

In this milestone, we will implement the utility functions for logging. The skeleton code contains a LOG() macro in utils.h which prints messages to the screen. LOG() calls are then spread throughout the server.c code. The goal of this milestone is to replace those LOG() calls with the logger() function residing in utils.c. Currently, the logger() function provides the same functionality as LOG(). Your tasks are as follows:
  1. Replace all LOG() macro calls by logger() calls in server.c,
  2. add four logger() calls in storage.c at appropriate locations, add comments to justify your choices (do this with in-line comments using //),
  3. add support for logger() to log to a file through a constant LOGGING defined both in client.c and server.c. The log filename should follow the format: Client/Server-yyyy-mm-dd-hh-mm-ss.log. This file is generated once per execution, which all logger calls made within an execution logged to the same file. The files should be generated in the same directory as the executables.
LOGGING should be an integer constant defined both for the client and server. The LOGGING parameter at client-side is used to control logging for all client-related code (which includes client.c and storage.c). When set to 0, logging is disabled. Set to 1, logging to stdout is enabled. When set to 2, logging is performed to a file that follows the specified format for the filename. For example, Client-2012-01-15-22-00-05.log is a log of an execution of the client made on January 15th, 2012 at 22:00:05. 

Appropriate locations for logging include critical parts of code which can potentially crash the system, such as library calls involving memory management, network message passing, etc. Other sections that may require logging include nested loops and chunks of new code that have not been proven reliable yet

Coding requirements

You are given a skeleton distribution that includes incomplete code for the storage server and client library, as well as makefiles and test suites.

For the most part, you can modify the code given to you as you like. However, you must observe the following constraints.

  • The code must be written in C. You cannot use any C++ features such as classes, or the iostream cin and cout objects.
  • The storage.h file should not be modified. This file declares the interface of the client library.
  • The server must be buildable by running make server in the src directory. The server executable must be called server.
  • The client library must be buildable by running make libstorage.a in the src directory. The client library must be called libstorage.a.

You're allowed to change any of the code (other than storage.h), add new source or header files, or modify the makefiles (subject to the constraints above).

Useful concepts

To complete this milestone, you will need to be familiar with the necessary functions to read from the user (stdin). The most popular way to do so is to use fgets(), which read a line of data from stdin into a buffer. You also need to know basic file I/O, in particular how to use fopen() to open a file and fprintf() to output a line into a file. To obtain the current date and time, refer to the time.h library.

Set up your code and development environment

Customize Your Environment

Here are some little tweaks to setup your development environment to your liking.

You can change the default editor that Subversion uses by setting the EDITOR environment variable. In the example below, you can replace pico with your favorite text editor.

> setenv EDITOR pico

Depending on how you login, you might not be able to use CTRL+C to terminate programs. You need to do this, for example, because the skeleton storage server given to you doesn't have a clean way to shut it down other than killing it. You might be able to get CTRL+C working by configuring the terminal properly:

> stty intr '^C'

If you don't like the default command line prompt, you can customize it by setting the prompt variable in CSH. Other shells will have different ways of changing the prompt. Here's an example.

> set prompt = "%n@%m:%~%# "

If you like the above customizations, you can put them in you shell's startup script (.cshrc for the CSH shell).

By default, you can use CTRL+H deletes the previous character in the console. If you would like to use backspace instead, change the terminal settings to set Backspace to CTRL+H.

Get the Skeleton Code

First, login into an EECG machine. ECF machines will not work!


Then, create a ece297 directory in your home directory to put everything in.

> mkdir ~/ece297
> cd ~/ece297
> tar zxf /cad2/ece297s/public/storage-skeleton.tgz

It is important that you do not change the location or name of the ece297 directory as it will be used when marking your code. You should rename the folder storage-skeleton to storage.

Browse the Skeleton Code

The files in the distribution are shown in the tree below. You can also find doxygen documentation of the code in Doxygen Documentation.

storage
|-- Makefile
|-- doc
|   |-- Makefile
|   `-- doxygen.conf
|-- src
|   |-- Makefile
|   |-- default.conf
|   |-- client.c
|   |-- encrypt_passwd.c
|   |-- server.c
|   |-- storage.c
|   |-- storage.h
|   |-- utils.c
|   `-- utils.h
`-- test
    |-- Makefile
    |-- Makefile.common
    `-- a1-partial
        |-- conf-duplicateport.conf  
        |-- conf-onetable.conf       
        |-- conf-twotables.conf
        |-- conf-threetables.conf  
        |-- Makefile
        |-- main.c
        `-- md5sum.check  

There are subdirectories for the source code (under src), documentation (under doc), and automatic test suites (under test). Each directory contains a makefile which you should at some point examine and try to understand. You can find some resources on the make utility in the Course Reader. For Milestone 1, you can ignore the doc and test directories, as they will only be used in future milestones.

The src directory contains the basic framework for you to implement your storage server and client library. Some notable files include:

  • storage.h which declares the client library functions such as storage_get() and storage_set() that you need to implement in subsequent milestones.
  • storage.c which contains incomplete implementations of the client library functions. However, it does include code to communicate with the server which may be instructive to look at.
  • server.c which includes an incomplete implementation of the storage server; as with storage.c, there is some socket programming code for communication here that might be instructive to look at.
  • client.c includes a very simple sample client that interacts with the server using the client library. For Milestone 1, the client should include a command-line shell to enter commands.
  • default.conf is a sample configuration file.
  • utils.h and utils.c contain some utility functions that you may find useful. In particular, the log function is located here.

The files above are well commented, and you should spend the time to try and understand the code.

Running the skeleton code

First compile the code:

> make

To run the server, give it the path of the configuration file as the first argument: 

> ./server default.conf

Note: you can edit default.conf to change the port used by your program, if the port is already being used.

If you want to run the server as a background process instead to free up your terminal, use this:

> ./server default.conf &

To run the client:

> ./client

If you ran the server as background process, use this to terminate it:

> killall ./server

Suggested development plan

Here are the tasks involved in this assignment.

  • Command-line shell
    • Displaying a menu
    • Reading input from the user
    • Calling the appropriate functions from the client library
  • Logging
    • Replacing LOG() calls by logger()
    • Adding four logger() calls in storage.c with justifications in comments
    • Add support for logging into file client.c and server.c.

Deliverables

By the software development deadline, you must hand in your software artifact that implement the coding requirements and include at least the following content.

  • You must hand in your code (you must make no change to storage.h file in the skeleton code).
  • The server and client must be set to log output to a file. (LOGGING = 2)

back to top