InkVault load test

verdict

Verdict

A VPS with 1 CPU core, 1 GiB of RAM and a 200 Mbit/s link handles InkVault. For a personal server, a family or a small group of up to about 100 users, the resources are more than enough.

During testing the server made no errors and lost no notifications, even under a load 10 times higher than it could process. This conclusion holds under these conditions:

• Prosody stores data in SQLite (storage = "sql"), as on the test bench
• Vaults hold up to 256 records, the default limit
• The VPS disk is an SSD

• 0errors in 307,977 operations
• 0lost notifications about new records
• 37 ms99% of requests were faster at 100 operations per second
• 30 MiBof memory used by Prosody with 50 connected clients

how

How InkVault uses the server

InkVault stores records on your XMPP server using PEP (XEP-0163), the standard mechanism for personal account data. Records are encrypted on the device; the server stores only the ciphertext and service fields: the record ID and the key version. InkVault needs no separate server or database.

What this means for the server: connecting a device is the most expensive operation, because the server sends all records at once. Saving is one small record and one database transaction with a disk write. Between connections the device sends no requests; changes arrive as notifications.

method

How it was tested

The load generator behaved like InkVault clients: it connected to Prosody over XMPP with TLS, downloaded vaults, and saved and read records. Record size and format match InkVault.

An operation is one client request: downloading the whole vault (8% of requests), saving a record (20%) or reading one record (72%).

latency

Response time under load

Up to 180 operations per second the response time barely changes. Beyond that, requests start waiting for each other, and at 260-280 operations per second the server reaches its limit: it cannot process more, and only the response time grows.

• 0.9 msmedian at 100 operations per second
• 37 ms99% of requests were faster
• 117 msslowest request in 5 minutes
• stableresponse time over 5 minutes

load

What loads the server

Prosody handles requests from all users one at a time, in a single thread. While a long operation runs, the other requests wait.

The longest operation is downloading the whole vault when a device connects. In the test it was 8% of requests, yet half of the server's working time.

Download time grows with the number of records. If the limit is raised to several thousand records, every connection of such a user will hold up the whole server for seconds.

vps

Are the VPS resources enough

Estimate for a VPS with 1 core, 1 GiB of RAM and a 200 Mbit/s link. 100 InkVault users, each with 2-3 devices, 10-30 connections and about 5 saves per day, up to 256 records.

• cpu

  CPU: up to 4 min per day

  The server spends 30 seconds to 4 minutes a day on InkVault requests, less than 0.3% of the time. A second core is not needed: Prosody runs in a single thread.

• memory

  Memory: 250-400 MB of 1 GiB

  System 150-250 MB, Prosody 30-60 MB, database cache about 70 MB. More than half of the memory stays free.

• disk

  Disk: about 70 MB database

  2.8 KB per record including indexes. One save writes up to 400 KB to disk: database pages and the transaction journal. For an SSD in personal use this volume is negligible.

• network

  Network: about 4 s after a restart

  In normal operation InkVault uses a fraction of a percent of the link. After a server restart, 300 devices download about 105 MB, which takes about 4 seconds on a 200 Mbit/s link.

• restart

  After a restart: 7-22 s

  This is how long it takes for all 300 devices to download their vaults again. The server processes the downloads one by one, so during these seconds it responds more slowly than usual. For 500 users it takes from 35 seconds to 2 minutes.

capacity

How many users such a VPS can handle

• users_1_100

  1-100 users

  Suitable, with a large margin.

• users_100_500

  100-500 users

  Probably suitable. Test on your VPS: a lot depends on its CPU speed.

• users_500_plus

  More than 500 users

  Not tested. A separate test on the target server is needed.

overload

What happens under overload

In the stress test the server received 10 times more requests than it could process.

• 3,000operations per second requested
• 264operations per second processed
• 0errors and lost notifications
• about 2 sresponse time instead of milliseconds

The server did not fail and lost no data: all requests completed, all notifications were delivered, and responses slowed down. After the load dropped, the accumulated queue was processed. A personal server's load is hundreds of times lower than this level.

setup

What to check on your server

Settings are in prosody.cfg.lua. Restart Prosody after changing them.

SQLite storage

The conclusions of this report apply to SQLite storage. By default Prosody stores data in files (internal); that option was not tested. To switch to SQLite:

storage = "sql"
sql = { driver = "SQLite3", database = "prosody.sqlite" }

This requires the LuaDBI package for SQLite: lua-dbi-sqlite3 on Debian and Ubuntu. Existing data can be moved with prosody-migrator.

Vault record limit

The limit is set by pep_max_items. The default of 256 matches InkVault's default vault limit. If you need more records, raise the limit:

pep_max_items = 10000

PEP module

The pep module is part of the standard Prosody configuration and is usually already enabled in modules_enabled. Testing was done on Prosody 13.0.7.

Disk speed

Save speed depends on how fast the disk confirms a write. A check for experienced users; the file is created next to the Prosody database and removed after the test:

If 99.00th in the sync percentiles block is 3000 usec or less, the disk is fine.

Rate limit

If the server limits connection speed (the limits module, for example 10 KB/s), the first import of 256 records takes about 50 seconds. This is the limit at work, not an error.

Backup

The InkVault vault lives in the Prosody database. You can back it up without stopping the server:

The database path depends on your installation.

database

SQLite and PostgreSQL

All conclusions in this report come from a test bench with SQLite and 100 users. SQLite is enough for a small server.

PostgreSQL is worth choosing for a server with hundreds of users or more. In a reference measurement on the same bench, the server wrote 4-5 times less to disk per save with PostgreSQL. Response time and CPU load were close to SQLite: Prosody runs queries to any database one at a time in its own thread, so the choice of database has little effect on the number of requests per second.