This chapter describes the functions in the libxmlrpc_server_pstream++ function library, which is part of XML-RPC For C/C++ (Xmlrpc-c). Also see General Library Information - C++
The libxmlrpc_server_pstream++ library provides C++ classes for use in a program that is an psuedo-XML-RPC server that uses the "packet stream" variation of XML-RPC.
When using libxmlrpc_server_pstream++, you must also use the libxmlrpc++ library. It contains additional facilities that an XML-RPC server needs but are general to XML-RPC and not specific to XML-RPC servers. Besides, the libxmlrpc_server_pstream++ classes depend on it.
Similarly, you will need the libxmlrpc_server++ library. It contains C++ classes for XML-RPC servers that are not specific to packet stream servers. And libxmlrpc_server_pstream++ classes depend on it.
Finally, you will need libxmlrpc_packetsocket, which provides the packet stream functions.
Note that Xmlrpc-c does not provide a C version of this facility.
All of the facilities in this chapter were new in Xmlrpc-c 1.11 (June 2007).
The classic Unix name for the file containing the libxmlrpc_server_pstream++ library is libxmlrpc_server_pstream++.a or libxmlrpc_server_pstream++.so. The classic linker option to cause the library to be linked into your program is -l xmlrpc_server_pstream++. These are hints; you'll have to modify this according to conventions of your particular platform. You'll also have to figure out where the library resides and how to make your linker look there for it.
You can use xmlrpc-c-config, specifying the "c++2" and "pstream-server" features, to find out what libraries to link. This is designed to be used in a build program such as a make file. When properly installed, it tells exactly how to link on your particular system.
The following libraries are prerequisites of libxmlrpc_server_pstream, so you'll need to link them in too:
Here is a complete example of a pseudo-XML-RPC packet stream server program. This program handles exactly one client connection, then exits. It expects to be passed the TCP connection for the client connection on a socket as Standard Input. That means this would be appropriate to use with an Inetd superserver. To turn this program into a standalone server that handles a series of client connections, you would just add some logic to accept TCP connections, such as Inetd has.
This example is for Xmlrpc-c 1.18 and later.
You can find a tested, buildable copy of this program in the examples directory of the Xmlrpc-c source tree as pstream_inetd_server.cpp. That's also how to find a version that works with older Xmlrpc-c.
#include <unistd.h>
#include <cassert>
#include <sys/signal.h>
#include <xmlrpc-c/base.hpp>
#include <xmlrpc-c/registry.hpp>
#include <xmlrpc-c/server_pstream.hpp>
using namespace std;
class sampleAddMethod : public xmlrpc_c::method {
public:
sampleAddMethod() {
// signature and help strings are documentation -- the client
// can query this information with a system.methodSignature and
// system.methodHelp RPC.
this->_signature = "i:ii"; // method's arguments are two integers
this->_help = "This method adds two integers together";
}
void
execute(xmlrpc_c::paramList const& paramList,
xmlrpc_c::value * const retvalP) {
int const addend(paramList.getInt(0));
int const adder(paramList.getInt(1));
paramList.verifyEnd(2);
*retvalP = xmlrpc_c::value_int(addend + adder);
}
};
int
main(int const,
const char ** const) {
// It's a good idea to disable SIGPIPE signals; if client closes his end
// of the pipe/socket, we'd rather see a failure to send a response than
// get killed by the OS.
signal(SIGPIPE, SIG_IGN);
try {
xmlrpc_c::registry myRegistry;
xmlrpc_c::methodPtr const sampleAddMethodP(new sampleAddMethod);
myRegistry.addMethod("sample.add", sampleAddMethodP);
xmlrpc_c::serverPstreamConn server(
xmlrpc_c::serverPstreamConn::constrOpt()
.socketFd(STDIN_FILENO)
.registryP(&myRegistry));
server.run();
} catch (exception const& e) {
cerr << "Something threw an error: " << e.what() << endl;
}
return 0;
}
A packet stream is a way to communicate XML-RPC messages that is far simpler than HTTP. XML-RPC specifies that messages are communicated by HTTP, so when you use a packet stream, you are not using XML-RPC, but something similar (We refer to it as "pseudo-XML-RPC").
XML-RPC For C/C++ defines a pseudo-XML-RPC protocol that is the same as XML-RPC except using a packet stream in place of HTTP. We call it "packet stream XML-RPC."
A packet stream is a form of communication that consists of a sequence of packets in each of two directions. A packet is a series of bytes. The packets of a stream may have varying sizes. A packet stream is best exemplified by a Unix sequential packet socket (type SOCK_SEQPACKET) (a little-known type that only some Unix-like operating systems offer).
Packet stream XML-RPC doesn't use a sequential packet socket because not all systems have them, but it wants to. Instead, it uses its own emulation of that kind of socket built over a traditional stream (SOCK_STREAM) socket. Each packet stream is uniquely associated with a stream socket (TCP) connection. See Packet Stream Protocol for technical detail on this protocol (not visible to a program using the facilities described in this chapter).
In packet stream XML-RPC, each XML-RPC message (i.e. XML-RPC call or XML-RPC response) is one packet.
An important difference between HTTP and a packet stream is that the packets of a stream are connected to each other -- they form a session or connection. In HTTP, each HTTP transaction is independent -- there is no concept of connection. This means that unlike in true XML-RPC, in packet stream XML-RPC, you can have a multi-RPC session.
That may not sound significant, because even in true XML-RPC you can build a connection above XML-RPC by defining login/logout RPCs. But it's better than that in one crucial way: when a program dies, the operating system automatically terminates any TCP connections to which it is party, which means when a packet stream XML-RPC client or server program dies, the OS under it ends the XML-RPC connection. The XML-RPC correspondent finds out immediately that the other party is dead.
Furthermore, you can arrange to have the connection end when there is network or OS failure (on the other end) as well. (See the TCP Keepalive section).
Note that packet stream XML-RPC is not a public standard. Only XML-RPC For C/C++ implements it. So you use it only in applications where you supply both client and server software. You can use Xmlrpc-c client facilities, in particular the xmlrpc_c::clientXmlTransport_pstream object class to build a suitable packet stream XML-RPC client.
An object of class xmlrpc_c::serverPstream is a packet stream XML-RPC server that can serve multiple client connections (not at the same time).
You supply, when you construct the object, a listening socket in the form of a socket file descriptor. This socket is already bound to a network address and in listen mode.
The object accepts TCP connections as they arrive via the listening socket, and for each connection, processes RPC calls as they arrive. It processes the connections and the RPCs serially, in a single thread.
You get much more flexibility with class serverPstreamConn, but have to write a little more code. With serverPstreamConn, for example, you can entertain multiple clients at once (by coding multiple threads) and you can have the server initiate a connection with a client rather than wait for a client to come to it.
This class was new in Xmlrpc-c 1.18 (March 2009). Before that, use serverPstreamConn.
Overview:
serverPstream::serverPstreamConn(constrOpt const& opt);
Examples:
Takes a listening socket on Standard Input:
xmlrpc_c::registry myRegistry;
...
xmlrpc_c::serverPstream server(xmlrpc_c::serverPstream::constrOpt()
.registryP(&myRegistry)
.socketFd(STDIN_FILENO)
);
Same, using an automatic pointer to manage registry lifetime and an internally accepted client connection:
xmlrpc_c::registryPtr myRegistryP(new xmlrpc_c::registry);
...
int acceptedFd;
acceptedFd = accept(...)
...
xmlrpc_c::serverPstream server(xmlrpc_c::serverPstream::constrOpt()
.registryPtr(myRegistryP)
.socketFd(acceptedFd);
);
This constructor uses the constrOpt paradigm to make specifying options easy and flexible, though technically there is just one C++ parameter.
The option methods are:
You must specify either registryPtr or registry, and not both.
You must specify either registryPtr or registry, and not both.
This option is mandatory.
This method causes the server object to accept TCP connections as they arrive over the associated listening socket, then execute RPCs as they arrive over those connections, essentially indefinitely. It does one connection and one RPC at a time; i.e. it accepts a connection, executes RPCs from that connection until the client closes the connection, then accepts the next connection, etc. For each connection, it receives an RPC call, then executes the RPC and sends the RPC response, then receives the next RPC call, etc.
Example:
server.runSerial();
Prototype:
void
serverPstream::runSerial();
void
serverPstream::runSerial(const int * interruptP);
If you don't want something this serial, use a serverPstreamConn object instead, and use multiple threads.
If runSerial is between RPCs, it returns immediately when it finds the interrupt flag *interruptP nonzero. It checks it (among other times) immediately after any signal handler returns, so you typically have a signal handler set it. Another common usage is to have another thread set the flag, then send the runSerial thread a signal, whose handler does nothing, to tell it to check the flag. The interrupt flag cannot interrupt an RPC that runSerial is in the middle of executing. In particular, it will not break out of an infinite loop or wait in your method object and it will not stop runSerial from taking as long as it takes to send the RPC response to the client. If you set the interrupt flag while runSerial is executing an RPC, runSerial will return as soon as it finishes that RPC.
If you use the form of runSerial that doesn't have interruptP, it does not return early.
After interrupting runSerial, you can invoke it again.
This method causes the server to stop running (i.e. a runSerial method call returns; any future runSerial call returns immediately).
It takes effect the next time a client connection closes.
You can call this from a method function, in which case it takes effect as soon as the connection via which the client invoked that RPC method closes.
You can call this from a signal handler. For example, you can make SIGTERM terminate the server.
Example:
server.terminate();
Prototype:
void
serverPstream::terminate();
An object of class callinfo_serverPstream is something that gets passed to your XML-RPC method (i.e. to the execute method of a xmlrpc_c::method object) to give it information about how the XML-RPC call arrived. It is an instance of xmlrpc_c::callInfo specific the serverPstream servers.
You can use this, among other things, to find out the IP address of the XML-RPC client.
This class was new in Xmlrpc-c Release 1.19 (June 2009). Before that, information about how the XML-RPC call arrived is not available to your XML-RPC method.
You don't construct or destroy a callInfo_serverPstream. Methods of serverPstream do that — you just follow a pointer to it and use it.
Member serverP is a pointer to the serverPstream object that is the XML-RPC server through which the call arrived.
serverPstream per se doesn't contain any members that are useful to your XML-RPC method, but derived class members might. For example, you might have defined a derived class of serverPstream called serverPstreamXyz and endowed it with information you want your XML-RPC method to have. Your XML-RPC method, then, need only do a dynamic cast of serverP * to serverPstreamXyz * and use that pointer to access your information.
This is the socket address of the client end of the client-server connection. Assuming it's a TCP connection, this gives you the IP address and TCP port number of the client. struct sockaddr is a Unix OS type; use it as you would with e.g. the POSIX accept() function.
This goes with clientAddr. Use it as you would use the corresponding address size value with the POSIX accept() function.
An object of class xmlrpc_c::serverPstreamConn is a single-connection packet stream XML-RPC server.
You supply, when you construct the object, an established TCP connection to the client, in the form of a socket file descriptor.
When you destroy the object, it shuts down the TCP connection, if the client has not done so already.
The object processes RPC calls as they arrive over the TCP connection. It processes them serially, in a single thread.
Overview:
serverPstreamConn::serverPstreamConn(constrOpt const& opt);
Examples:
Takes an established client connection on Standard Input:
xmlrpc_c::registry myRegistry;
...
xmlrpc_c::serverPstreamConn conn(xmlrpc_c::serverPstreamConn::constrOpt()
.registryP(&myRegistry)
.socketFd(STDIN_FILENO)
);
Same, using an automatic pointer to manage registry lifetime and an internally accepted client connection:
xmlrpc_c::registryPtr myRegistryP(new xmlrpc_c::registry);
...
int acceptedFd;
acceptedFd = accept(...)
...
xmlrpc_c::serverPstreamConn server(
xmlrpc_c::serverPstreamConn::constrOpt()
.registryPtr(myRegistryP)
.socketFd(acceptedFd);
);
This constructor uses the constrOpt paradigm to make specifying options easy and flexible, though technically there is just one C++ parameter.
The option methods are:
You must specify either registryPtr or registry, and not both.
You must specify either registryPtr or registry, and not both.
This option is mandatory.
This method causes the server object to execute RPCs as they arrive over the associated connection, until the client terminates the connection.
Example:
server.run();
extern int interruptFlag;
class myCallInfo : public xmlrpc_c::callInfo {
public:
myCallInfo::myCallInfo(string const& myname) :
myname(myname) {}
string myname;
}
myCallInfo callInfo("George");
interruptFlag = 0;
server.run(&callInfo, &interruptFlag);
Prototype:
void
serverPstreamConn::run();
void
serverPstreamConn::run(const int * interruptP);
void
serverPstreamConn::run(xmrpc_c::callInfo * callInfoP,
const int * interruptP);
For finer control over the processing, use runOnce or runOnce instead.
If run is between RPCs, it returns immediately when it finds the interrupt flag *interruptP nonzero. It checks it (among other times) immediately after any signal handler returns, so you typically have a signal handler set it. Another common usage is to have another thread set the flag, then send the run thread a signal, whose handler does nothing, to tell it to check the flag. The interrupt flag cannot interrupt an RPC that run is in the middle of executing. In particular, it will not break out of an infinite loop or wait in your method object and it will not stop run from taking as long as it takes to send the RPC response to the client. If you set the interrupt flag while run is executing an RPC, run will return as soon as it finishes that RPC.
If you use the form of run that doesn't have interruptP, it does not return early.
The callInfoP argument points to an object that run passes to your XML-RPC method execute method. It is opaque to Xmlrpc-c. Use this object to pass information about the context in which the server is running, if that should affect what RPCs do. For example, you might include the IP address of the XML-RPC client.
If you use the form of run that doesn't have callInfoP, you have no way to pass context information to your XML-RPC method object. callInfoP was new in Xmlrpc-c 1.19 (June 2009).
This method was new in Xmlrpc-c 1.18 (March 2009). Before that, just use runOnce in a loop.
This method causes the server object to process one RPC (waiting for one if necessary). It waits for the next packet to arrive from the client, executes the indicated RPC (which includes sending a packet as a response), and returns.
If before a call arrives the client terminates the connection, the method returns at that time, with an indication that no more RPCs will be forthcoming.
Example:
for (bool clientHasDisconnected = false; !clientHasDisconnected;)
server.runOnce(&clientHasDisconnected);
Prototype:
void
serverPstreamConn::runOnce(bool * eofP);
void
serverPstreamConn::runOnce(const int * interruptP,
bool * eofP);
void
serverPstreamConn::runOnce(xmlrpc_c::callInfo * callInfoP,
const int * interruptP,
bool * eofP);
runOnce returns *eofP true if the client terminated the connection before the method received an RPC call (and therefore the method didn't receive an RPC call). Otherwise, it returns *eofP false.
Ordinarily, you invoke this method in a loop until either it reports that the client has disconnected or your program decides to quit on its own, then you destroy the serverPstreamConn object.
When runOnce is waiting for the RPC call to arrive, it returns immediately if it finds the interrupt flag *interruptP nonzero. It checks it immediately after any signal handler returns, so you typically have a signal handler set it. Another common usage is to have another thread set the flag, then send the runOnce thread a signal, whose handler does nothing, to tell it to check the flag. Another time runOnce checks the flag is before it does anything. The interrupt flag has no effect after runOnce has started executing an RPC.
If you use the form of runOnce that doesn't have interruptP, it does not return early.
interruptP was new in Xmlrpc-c 1.14 (March 2008).
runOnceNoWait is the same thing except that it returns immediately if no RPC has been received.
This method causes the server object to process one RPC, if a packet is immediately available. If one is, it executes the indicated RPC (which includes sending a packet as a response), and returns. Otherwise, it returns immediately.
If the client has terminated the connection, the method returns immediately, with an indication that no more RPCs will be forthcoming.
Example:
for (bool clientHasDisconnected = false; !clientHasDisconnected;) {
sleep(1);
server.runOnceNoWait(&clientHasDisconnected);
}
Prototype:
void
serverPstreamConn::runOnceNoWait(bool * eofP);
void
serverPstreamConn::runOnceNoWait(bool * eofP,
bool * didOneP);
What it means for a packet to be immediately available is generally that the operating system under the client has received one (and it's sitting in an OS socket buffer). Specifically, it means either that the client has previously read a whole packet from the OS socket underlying the packet socket on which the client is built, or it can read (or finish reading one) now, while telling the OS it doesn't want to wait for data.
runOnceNoWait returns *eofP true if the server has not received any RPC calls that haven't been executed yet, and the client has terminated the connection. Otherwise, it returns *eofP false.
runOnceNoWait returns *didOneP true if the client has received an RPC call (which means runOnceNoWait processed it). Otherwise, it returns *didOneP false.
runOnceNoWait never executes more than one RPC.
Ordinarily, you invoke this method in a loop, which also does other things unrelated to XML-RPC until either the method reports that the client has disconnected or your program decides to quit on its own, then you destroy the serverPstreamConn object.
You may want to use this in conjunction with the POSIX select, etc. Use select to determine when data is available to read on the OS socket on which the client is built. When you find that data is available on the socket, call runOnceNoWait to process the incoming RPC. Note that data received on the socket doesn't necessarily mean an entire RPC call was received; the only way to know that is to run runOnceNoWait and check the *didOneP result.
runOnce is the same thing except that it waits for an RPC call to arrive.
runOnceNoWait was new in Xmlrpc-c 1.18 (March 2009).
You can use the XMLRPC_TRACE_XML environment variable to see the calls and responses from the libxmlrpc_server registry's point of view. With a packet stream server, there is a simple relationship between packet traffic and registry traffic, so this is essentially the same thing as a packet trace.
The XMLRPC_TRACE_PACKETSOCKET environment variable traces at a lower level, showing you packets sent and received on the packet socket as well as raw bytes (including packet socket escape sequences) read. This feature was new in Xmlrpc-c 1.41 (March 2015).
This section describes the protocol that packet stream XML-RPC uses to transport XML-RPC messages. The protocol itself is independent of XML-RPC -- it merely transports arbitrary packets.
The goal of the protocol is to emulate a Unix sequential packet socket (type SOCK_SEQPACKET), but work on more systems than have packet stream socket function. The idea is a hybrid of two far more common socket types: stream (type SOCK_STREAM) and datagram (type SOCK_DGRAM). We want to have connections and reliable, in-order delivery like with a stream socket, but identified and indivisible packets like a datagram socket. Note that a stream socket handles a stream of bytes, not packets. A datagram socket transfers whole packets, such that when you write to the socket, you write exactly one packet (if any), and when you read from a socket, you get exactly one packet (if any).
The packet stream protocol is based on a Unix stream socket. A Unix stream socket is usually an interface to a TCP connection. We will refer to the stream of bytes transported by this stream socket as "the byte stream."
Example:
P a c k e t O n e <esc> P K T
E s c a p e : <esc> E S C . <esc> P K T
(white space and line breaks in the example are for display only).
In the byte stream, the ASCII code for the Escape character (<esc>), which is 0x1B, is an escape character -- it introduces a 4 byte protocol control word. We specify the 4 bytes of a control word with ASCII characters such that the 4 bytes are the ASCII encoding of 4 ASCII characters. The first one is of course always <esc>.
The byte stream is partitioned into packet frames. The first packet frame begins with the first byte of the byte stream. Each packet frame ends with a packet delimiter, which is the control word <esc> P K T . It is a protocol error for the byte stream to end anywhere except immediately after a packet delimiter or at the very beginning of the byte stream.
Within a packet frame, every byte that is not part of a control word represents a byte of the packet with the same value, in the same order.
Within a packet frame, the control word <esc> E S C represents the byte 0x1B (ASCII Escape) in the packet.
It is a protocol error for the byte stream to contain any control word other than the two mentioned here.
While ASCII is used to specify some of the protocol, this does not imply that the protocol is about ASCII or text or characters. A packet consists of arbitrary bytes.
The libxmlprc_packetsocket library, part of Xmlrpc-c, provides facilities for having a packet stream conversation. This library does not involve XML-RPC.
The TCP keepalive function is important to users of Packet Stream XML-RPC. One of the big advantages of Packet Stream XML-RPC over regular XML-RPC is that a client or server can know when its peer dies (or, equivalently, becomes unreachable over the network). TCP keepalive is part of what makes that possible.
TCP keepalive is not in effect for a default TCP connection. Such a connection ends only when the OS on one side acts to end it. Usually, that's because the OS user of the socket closed it, but it can also be because the OS forcibly closed the socket because the user process terminated. But what if the OS itself or something below it dies? For example, the OS could hang, or the power could fail on the computer that's running it, or a network cable could get cut. In that case, the default TCP connection can live indefinitely. The OS on the surviving end will not notice anything wrong until it tries to send data and, after retries and waits, fails to get an acknowledgement.
TCP keepalive allows the surviving end to find out more timely that the other end is gone. To activate it, you set the SO_KEEPALIVE socket-level option on the socket, using the POSIX setsockopt() function. On many systems (including Linux), you can control the specifics of the keepalive function with the TCP-level socket options (which you set also with setsockopt()) TCP_KEEPIDLE, TCP_KEEPINTVL, and TCP_KEEPCNT.
With the keepalive function, the OS periodically sends empty segments to the other side. If it fails to get an acknowledgement, it declares the peer dead and closes the connection. Assuming the TCP socket underlies an Xmlrpc-c packet socket, that packet socket closes as well, and any attempt to read from it fails. If a process is in the middle of method that waits for data to arrive, that method returns a failure at that time.
In Xmlrpc-c, you create a packet socket out of a fully-formed TCP socket, so it is your choice whether to have keepalive active on it, and the specific parameters of it. You nearly always do want it. If your application doesn't need to know that the peer still exists, then it is probably the kind of application that would work better with normal connectionless XML-RPC.
The TCP Nagle function is important to some users of Packet Stream XML-RPC. This is a function whereby the OS delays transmission of TCP data. To reduce utilization of TCP resources such as network link time, the OS may hold off on sending data to wait for additional data to send. If the application requests the sending of more data soon enough, the OS sends all of the data in a single packet, thus saving resources compared to sending multiple packets. The policy by which the OS decides when to wait and how long is called the Nagle Algorithm, named after its inventor. The process of doing the waiting is called nagling.
The OS does nagling by default, but you can tell it not to (disable nagling) for a particular TCP socket. When nagling is disabled, the OS sends IP packets as soon as it can. You control nagling with the TCP-level socket option (which you set with setsockopt()) TCP_NODELAY.
Nagling is not as valuable as it seems, because it conserves resources mainly on systems where resource contention isn't a problem. If a network link is actually congested, the OS will not be able to send a packet as soon as the application asks it to send some data, so it will end up accumulating small send requests into large packets anyway. Nagling causes the OS to refrain from sending only when there are resources available to send. Still, Nagling can be a useful hack to allocate resources in some simple systems.
An Xmlrpc-c packet server ordinarly makes three send requests (socket writes) per RPC call or response (packet header, packet body, packet trailer). A typical RPC call or result fits in one IP packet. So without nagling, on an unloaded network, you would see 6 IP packets per RPC, whereas with nagling, you would see 2.
On a system I measured, the Nagle Algorithm slowed a continuous serial RPC process down to 12 RPCs per second.
In Xmlrpc-c, you create a packet socket out of a fully-formed TCP socket, so it is your choice whether to have the OS perform nagling or not.