COMP30023 Remote Procedure Call Project Assessment

1 Project Overview

Remote Procedure Call (RPC) is a crucial technology in distributed computing that enables software applications to communicate with each other seamlessly over a network. It provides a way for a client to call a function on a remote server as if it were a local function call. This abstraction allows developers to build distributed systems and applications that span multiple machines and platforms.

In this project, you will be building a custom RPC system that allows computations to be split seamlessly between multiple computers. This system may differ from standard RPC systems, but the underlying principles of RPC will still apply.

Your RPC system must be written in C. Submissions that do not compile and run on a Linux cloud VM, like the one you have been provided with, may receive zero marks. You must write your own RPC code, without using existing RPC libraries.

2 RPC System Architecture

Your task is to design and code a simple Remote Procedure Call (RPC) system using a client-server architecture. The RPC system will be implemented in two fifiles, called rpc.c and rpc.h. The resulting system can be linked to either a client or a server. For marking, we will write our own clients and servers, and so you must stick to the proposed API carefully.

For testing purposes, you may run server and client programs on the same machine (e.g., your VM).

3 Project Details

Your task is to design and code the RPC system described above. You will design the application layer protocol to use. A skeleton is provided which uses a simple application programming interface (API). When we assess your submission, we will link our own testing code using the same RPC system API; what you will be assessed on is rpc.c (and any other supporting files compiled in by your Makefile).

Note that implementing the API will require you to use sockets. This uses material covered in the lectures after the project is released. There is plenty to do on the project before you need to use sockets, so please do not say that you cannot start because you have not yet learned about sockets.

3.1 API

The basic API implemented by rpc.c consists of the following data structures and functions.

3.1.1 Data structures

The API will send and receive data structures of the form:

typedef struct {
int data1;
size_t data2_len;
void *data2;
} rpc_data;

where data1 is an integer to be passed to the other side, and data2 is a block of bytes (of length data2_len) to be sent. The purpose of data1 is to allow simple functions that only pass an integer to avoid memory management issues, by setting data2_len=0 and data2=NULL. Your protocol can limit data1 to being no more than 64 bits.

Note that size_t depends on the architecture, and the sender and receiver can have different architectures. Think how this will affect your protocol.

The handler that implements the actual remote procedure will have the signature:
rpc_data *procedure (rpc_data *d);

That is, it takes a pointer to an rpc_data object and returns a pointer to another rpc_data object. This function will dynamically allocate memory with malloc for both the rpc_data structure and its data2 field. It is the responsibility of the RPC system to free those after use.

The state of the client and server will be in data structures that you defifine. These are declared in rpc.h as:
typedef struct rpc_client rpc_client;
typedef struct rpc_server rpc_server;
and you should provide the actual struct defifinitions in rpc.c. These are returned by initialization functions (rpc_init_client and rpc_init_server, and passed to all other functions. ,/p>

3.1.2 Server-side API

rpc_server *rpc_init_server (int port)
Called before rpc_register. Use this for whatever you need. It should return a pointer to a struct (that you define) containing server state information on success and NULL on failure.

int rpc_register (rpc_server *srv, const char *name, rpc_data* (*handler)(rpc_data*)) At the server, let the subsystem know what function to call when an incoming request is received. It should return a non-negative number on success (possibly an ID for this handler, but a constant is fine), and -1 on failure. If any of the arguments is NULL then -1 should be returned.

Think:

1. What are the valid characters in name? (We will only test printable ASCII characters between 32 and 126.)
2. How does the server know the length of name? (We will not test names longer than 1000 bytes.)
3. Should there be a minimum length for name? (We will not test for empty names.)

If there is already a function registered with name name, then the old function should be forgotten and the new one should take its place.

To get full marks, you should be able to register at least 10 functions. You can still get most of the marks as long as you can register one function, so implement that first.

void rpc_serve_all (rpc_server *srv)
This function will wait for incoming requests for any of the registered functions, or rpc_find, on the port specified in rpc_init_server of any interface. If it is a function call request, it will call the requested function, send a reply to the caller, and resume waiting for new requests. If it is rpc_find, it will reply to the caller saying whether the name was found or not, or possibly an error code.

This function will not usually return.
It should only return if srv is NULL or you’re handling SIGINT (not a requirement).

3.1.3 Client-side API

rpc_client *rpc_init_client (const char *addr, int port)
Called before rpc_find or rpc_call. Use this for whatever you need. The string addr and integer port are the text-based IP address and numeric port number passed in on the command line.

The function should return a non-NULL pointer to a struct (that you defifine) containing client state information on success and NULL on failure.

void rpc_close_client (rpc_client *cl) Called after the fifinal rpc_call or rpc_find (i.e., any use of the RPC system by the client).
Use this for whatever you need; it should at least free(cl).
If it is (mistakenly) called on a client that has already been closed, or cl == NULL, it should return without error.(Think: How can you tell if it has already been closed?)

rpc_handle *rpc_find (rpc_client *cl, const char *name)
At the client, tell the subsystem what details are required to place a call. The return value is a handle (not handler) for the remote procedure, which is passed to the following function.

If name is not registered, it should return NULL. If any of the arguments are NULL then NULL should be returned. If the fifind operation fails, it returns NULL.

rpc_data *rpc_call (rpc_client *cl, rpc_handle *h, const rpc_data *data)
This function causes the subsystem to run the remote procedure, and returns the value.
If the call fails, it returns NULL. NULL should be returned if any of the arguments are NULL. If this returns a non-NULL value, then it should dynamically allocate (by malloc) both the rpc_data structure and its data2 field. The client will free these by rpc_data_free (defined below).

The skeleton gives an example of how these functions can be used.

3.1.4 Shared API

void *rpc_data_free (rpc_data* data)
Frees the memory allocated for a dynamically allocated rpc_data struct.
Note that there is a reference implementation of this function in the skeleton code.

3.2 Planning task

When you are designing the protocol, ask yourself the following questions. Think of the answer for this particular project, and separately for the case of a “real world” RPC server.

Put the answers in a plain text fifile answers.txt, beginning with your name, login ID and student ID. These, together with protocol description below, are worth 1 mark.

1. Should the server accept calls from everyone, or just a subset of users?
2. Should authentication etc. be provided by the RPC framework, or by the functions that use the RPC framework?
3. What transport layer protocol should be used? What are the trade-offs?
4. In which function(s) should the socket(s) be created?
5. Should rpc_client and rpc_server be allocated dynamically or statically? What are the implications for the client and server code?
6. What happens if one host uses big-endian byte order and the other uses little-endian? How does that relate to “network byte order”?

3.3 Protocol

You should design and document a simple application layer protocol for this RPC system. (It can be very simple.) Describe it in the fifile answers.txt, in enough detail that someone else could implement the protocol. Together with the questions above, this is worth 1 mark.

Note that size_t is system-dependent. You will need to choose a system-independent way to encode the size of the data block in the packet. You can use a universal encoding, like Elias gamma coding (see Wikipedia) which can specify arbitrarily long strings, or you can use a fifixed sized fifield, which is simpler to decode but limits the size of the string. Explain your choice in your protocol description.

In all test cases, len < 100>

If data2_len is too large to be encoded in your packet format, the relevant function in rpc.c should print "Overlength error" to stderr and return an error. The protocol should specify error responses for routine failures, such as a request for a procedure that does not exist.

The protocol should work correctly even if there are IP layer packet loss and duplication.

The protocol should handle the fact that IP packets have a maximum allowed size.

Decide what transport layer protocol to use. (You will almost certainly choose TCP, but briefly mention the pros and cons of alternatives.)

The transport layer protocol should run on top of IPv6.

3.4 Test harness

Code isn’t complete until it has been thoroughly tested.

If your Makefile produces executables rpc-server and rpc-client, then they will be executed by the CI with the command line arguments shown below, with the results of stdout and stderr included in the test transcript.

No marks are allocated to rpc-server and rpc-client, either in execution or in code quality.

To run your server program on your VM prompt, type:

• /rpc-server -p &
• /rpc-client -i -p

where:

• The & tells the operating system to run the server in the background.
• ip-address is the IPv6 address of the VM on which the server is running.
• port is the TCP (or other transport layer) port number of the server.

The server is expected to listen for incoming connections on the port passed via command line arguments, on any of the hosts IPv6 network addresses.

4 Stretch goal: non-blocking performance

Many RPC operations, such as database look-ups, take a substantial time to complete. Instead of making each request wait for all those before it to complete, it is possible to continue to accept and start processing new requests while previous requests are executing, with each call returning as soon as it is finished. The simplest way to do this is with multiple threads: each time a request (or connection) is received, a new thread is spawned to execute the procedure, and is destroyed once the result has been sent back to the caller. Alternatives include creating new processes with fork(2), or using select(2).

If you implement non-blocking, include the line “#define NONBLOCKING" in your code. Otherwise, this functionality will not be tested/marked.