
A short description of the VIP1710s IPC algorithm

Hans-Werner Hilse <hilse@web.de>, 2010
Version: 2010-09-21

This document may be used by whomever for whatever she/he
wants, given that she/he knows what she/he is doing and
this licence is kept in effect on derived versions.
(the know-what-you're-doing licence)

This document should be downloadable at
http://hilses.de/vip1710/libvipipcstack/README-IPC.txt
in its most current version.

Product names in this document belong to their respective authors, I do not
claim them but use them to name the corresponding products.



TOC:

1. Preface
2. Communication Infrastructure: Unix Domain Sockets
3. Communication Agents
4. Language: Message (De-)Construction
5. Misc



1. Preface

I'm still playing with a piece of hardware that an electronics retailer here in
Germany offered for 5 EUR, giving no promises for whatever usage scenario. It's
a small little IPTV set top box (STB), able to decode MPEG1 and MPEG2 video
(maybe more, it also contains a DSP that should allow for more compression
schemes) and MP2 audio. The Box is based on the ATI Xilleon 210 SoC, which
operates as the primary CPU on these Boxes (with a MIPS 4kec synthesized core
operating at 300MHz). 

Unfortunately, specs for the chipset are somewhat closed. The WWW offers some
few instances of documentation pieces that leaked, but they are mostly for
other Xilleon based designs and even in sum they do not give a very good idea
on how to drive the platform with your own software.

In case of the STB the author of this document bought, a VIP1710, designed by
Kreatel, later bought by Motorola, a closed-source userland software stack
drives the functionality of the Xilleon platform. It consists of multiple
software components, which are introduced in section 3 below. These software
components communicate with each other and offer their functionality to the
other components.

An easy way to make some use of the platforms functionality, especially
regarding its media streaming features, seemed to hook into this software
stack, at least, until some major parts of it can be re-engineered. So the
author started to examine what's going on between the software components.
This document is the intellectual result of that work and is accompanied by
some software: There's a unix domain socket traffic dumper which was named
"sockdump" (find it at http://hilses.de/sockdump/) and finally a C
implementation of the communication mechanism, "libvipipcstack" (you can find
it here: http://hilses.de/vip1710/libvipipcstack/).

The communication mechanism will be further referred to as "IPC", the
well-known acronym for Inter-Process-Communication.


2. Communication Infrastructure: Unix Domain Sockets

When looking at the software as it runs, running "lsof" and "strace", it was
very easily determined, that the communication is based on passing messages on
sockets. It uses Unix Domain Sockets, which share the socket API e.g. with IP
sockets, which are far more popular. Unix Domain Sockets operate on a
machine-level, though. There are processes that operate as servers, allowing
one or more connections from connecting clients. You may be tempted to compare
them to Named Pipes, but that would leave out the more general features of Unix
Domain Sockets. In comparison to (named) pipes, Unix Domain Sockets have a
broader idea of connection management. Plus, there's better buffer management,
multiple connections on the same socket name, better connection state
management etc.

The vendor software stack uses an IPC mechanism that uses sockets that are
being created in /tmp and generally have names with a "ipc_" prefix and a
random-like appendix.

Within the IPC protocol, these sockets are sometimes multiplexed and are
identified internally with a suffix "#" plus a number, starting at 0. This
suffix is not taken into action when creating, listening on or connecting to
the sockets. So in the filesystem, the sockets look like this:

  /tmp/ipc_A2b4C8
  /tmp/ipc_d9E75f

while they are referred within the IPC communication as

  /tmp/ipc_A2b4C8#0
  /tmp/ipc_A2b4C8#3
  /tmp/ipc_d9E75f#0

In this example case, the first socket is multiplexed within the IPC stack to
two "virtual" sockets. Connections to the same "physical" socket are assigned
to the "virtual" sockets by looking in the IPC messages: they carry an address
for the "virtual" socket they are meant to get sent to. More on this in section
4 below.


3. Communication Agents

I'll give a short overview on some parts of the userland software stack here to
give an idea of the software parts and their functionality:

procman:
	This is the first process that gets started after system
	initialization. It does the following:

	* manage _services_. _services_, of which other software components do
	  provide one or more, are started by and/or registered at the
	  procman process. It offers the NameService, which is an information
	  broker used to register and query service<->socket name associations.

	* the ProcessService allows to have procman start and supervise other
	  processes. All processes (services) started/supervised by procman
	  are run with individual uids and a given fixed gid. procman itself
	  runs as the root user.

	* the TimeService seems to simply give access to clock data
	
halserver:
	Gets run by procman before doing anything else. It provides some
	services that manage access to system resources. It runs as the root
	user.

	halserver manages, among a lot of other things, the platform watchdog
	timer and offers an IPC interface for "feeding" the watchdog.

sysman:
	Operates close to halserver and procman. It manages the software stack
	but uses procman to start and supervise the according processes.

	sysman is also in charge of feeding halserver's watchdog interface.

softwareservice:
	Seems to manage firmware as a whole entity. Not yet analyzed very
	closely, but it asks halserver for the crypto keys for the firmware.

appman:
	Manages applications, which can be started on demand or via URI or MIME
	types. Other services register to this in order to get notified as
	applications start up or in order to request applications to get
	started. It also does supervision of application processes by sending
	them pings and waiting for replies. It uses procman services to
	actually start or kill programs.

infobroker:
	Is a service, providing the InformationService which manages key-value
	associations, i.e. provides a common database. It is used for permanent
	storage (persistent configuration) and stores that in
	/flash/settings2.xml. It also maintains memory-only associations
	(volatile) which are also being used for pro-active status signalling
	for other applications (e.g. the current network address is stored
	there). It also offers a notification interface that triggers IPC calls
	upon changes to certain keyed data.

netconfig:
	Well, even the name makes obvious what it does.

displaymanager:
	Not yet very carefully analyzed, but it might just be a big, albeit not
	very mighty, X window manager. It queries for window sizes and it seems
	that's about it. Seems to trigger if a mouse pointer is visible, too...

X:
	If you don't know on your own, you're not allowed to read this.

mediamanager:
	Seems to manage stream sources. It works closely together with the
	streamer process. It gets some important resources from halserver
	(which makes them unavailable for other clients), like VideoMixer,
	VideoOutput and AudioMixer. It then offers its own services on top of
	these which are (normally) accessed by streamer and tv.

streamer:
	This is kind of a media player backend. It has support for transports
	and container formats. It uses halserver's TsParser resource for doing
	MPEG stream parsing (as it seems, not just TS but also PS and PES) with
	the platform's hardware features. The streamer process supports
	different transport schemes which are differentiated by URI schemes:
	* HTTP: http://<ip or hostname>[:port][/<path>]
	* RTPS: rtps://<ip or hostname>[:port][/<path>]
	* IGMP: igmp://<multicast-ip>:port
	* UDP: udp://<local ip>:<port>
	* RTP: rtp://<local ip>:<port>
	* TCP: tcp://<remote ip>:<port>
	* FILE: file:///path
	* LSTRM: lstrm:///path  (Unix Domain Sockets)
	* LDGRAM: ldgram:///path  (Unix Domain Sockets)

tv:
	This is the media player frontend. It needs X+GTK for doing the status
	displays.

(and a bit more I'm currently too lazy to describe right now. Check back in a
few weeks if you cannot manage to find out on your own.)


4. Language: Message (De-)Construction

So, that was a lengthy discussion of what else is there, but it was promised to
describe the IPC mechanism, right? So it's time to see how the actual messages
are formatted. It's not very complicated, so this section is a short one :-)

#1: messages are stacks: Consider every message as a stack of data.

#2: datatypes:

	there are Bytes (which are also being used to store booleans),
	Ints and Strings as far as I could see.
	Byte: well, not much to say about this.
	Int: 32bit, lower endian when you look at the message. Big endian if
		you're reading the messages backwards because I told you
		they're stacks :-)
	String: Can contain \0, so they are suffixed (i.e. "prefixed in stack
		order") with an Int value (as described above) indicating their
		length. They do not have to end with \0, but in practice, most
		of them do.

#3: message header: all messages start with an Int giving the overall message
length.

#4: procedure calls: a lot of methods are procedure calls. These end (i.e. have
at the top of their stack) in (reading from end to start)
	- a byte which is usually 0 (could not yet make sense of the exceptions
	  to that rule...)
	- a string (encoded as described above) giving the procedure name
	- a string (...) giving the name of the called interface
	- a string (...) giving the name of the virtual socket name (see
	  section 2)

#5: replies: do not have the procedure call signature described above, but
usually and on a single byte value indicating if there was an error (1) or not
(0).

Structure overview:
RPC:
	<Int(message length)>
	<payload/arguments>
	<String(virtual socket)>
	<String(interface)>
	<String(procedure name)>
	<Byte(?)>

Reply:
	<Int(message length)>
	<payload>
	<Byte(error)>

If there are errors, they are commonly encoded (i.e. the reply looks) like
this:
	<Int(message length)>
	<Int(error code)>
	<String(error message)>
	<Byte(error = 1)>

When reading messages from a network socket, you can start by reading an Int -
the message size -, create a buffer of that size and then read message size
minus sizeof(Int)=4 further bytes.

Received messages should be processed by pop'ing from the stack.


5. Misc

This has been just a very small description on what's going on. If you're
interested in the interfaces and how to use them, you should look at the
libvipipcstack mentioned in section 1. It has prototypes for a lot of
identified Interface/Procedure combinations and brings examples on how to use
them in your own software. This is an ongoing approach and if you're interested
in this, reading C header files should really be an ability of yours. That
said, look int vipipc_prototypes.h in said library to see what already got
identified. Use the software description above in section 3 in order to make
sense of how the components interact. And then there's always: Trying, seeing
what happens.

No guarantees whatsoever are taken for the software and the information
provided here. It may all be wrong, we may all live in different realities. I
don't care since it works for me and you did not pay me, after all.

