Benchmark

Linqdb benchmark uses Stack Overflow dataset as of late 2019 (roughly 18 000 000 questions).
Inspired by: Ravendb's blog
Data from: Stackexchange archive
Benchmark code: Linqdb benchmark

Data model: Data.cs

Code:

1. Data import - imports Stack Overflow's data - Program.cs
2. Signups per month - SignupsPerMonth.cs
3. User activity per month - ActiveUsersByMonth.cs
4. Tag activity per month - TagsPerMonth.cs
5. Tag's info - TagsInfo.cs
6. Tag activity per month, fast version - TagsPerMonthFast.cs
7. Tag's info, fast version - TagsInfoFast.cs


Embedded db, SSD disk - Windows:
1. Data import: 502 min, db size 67 Gb
2. Signups per month: 47 sec
3. User activity per month: 421 sec
4. Tag activity per month: 1164 sec
5. Tag's info: 1408 sec
6. Tag activity per month, fast version: 330 sec
7. Tag's info, fast version: 535 sec

Server db, server and client in the same geographic region, client on Windows, server on SSD Linux server:
1. Data import: 315 min, db size 67 Gb
2. Signups per month: 45 sec
3. User activity per month: 383 sec
4. Tag activity per month: 889 sec
5. Tag's info: 1110 sec
6. Tag activity per month, fast version: 409 sec
7. Tag's info, fast version: 602 sec



Benchmark to model many concurrent users reading entire posts:
SERVER Linux:

1000 jobs, 1 thread
Total: 49343 ms
Total: 1000, errors 0, min 5 ms, max 148 ms, avg 26,901 ms

1000 jobs, 10 threads
Total: 27590 ms
Total: 1000, errors 0, min 5 ms, max 308 ms, avg 24,108 ms

1000 jobs, 100 threads
Total: 26176 ms
Total: 1000, errors 0, min 4 ms, max 328 ms, avg 22,644 ms

***************************************************************

10000 jobs, 1 threads
Total: 160485 ms
Total: 10000, errors 0, min 3 ms, max 149 ms, avg 13,3575 ms

10000 jobs, 10 threads
Total: 61329 ms
Total: 10000, errors 0, min 3 ms, max 343 ms, avg 21,1137 ms

10000 jobs, 100 threads
Total: 62273 ms
Total: 10000, errors 0, min 4 ms, max 401 ms, avg 24,1838 ms

***************************************************************

100000 jobs, 10 threads
Total: 399296 ms
Total: 100000, errors 0, min 4 ms, max 1301 ms, avg 31,87717 ms


===============================================================

EMBEDDED Windows:

1000 jobs, 1 thread
Total: 105663 ms
Total: 1000, errors 0, min 11 ms, max 635 ms, avg 66,149 ms

1000 jobs, 10 thread
Total: 46167 ms
Total: 1000, errors 0, min 12 ms, max 500 ms, avg 72,638 ms

1000 jobs, 100 thread
Total: 49621 ms
Total: 1000, errors 0, min 10 ms, max 625 ms, avg 72,583 ms

***************************************************************

10000 jobs, 1 threads
Total: 441151 ms
Total: 10000, errors 0, min 2 ms, max 641 ms, avg 40,5309 ms

10000 jobs, 10 threads
Total: 120670 ms
Total: 10000, errors 0, min 7 ms, max 3008 ms, avg 78,2662 ms

10000 jobs, 100 threads
Total: 111750 ms
Total: 10000, errors 0, min 4 ms, max 865 ms, avg 67,218 ms

***************************************************************

100000 jobs, 10 threads
Total: 758725 ms
Total: 100000, errors 0, min 2 ms, max 1972 ms, avg 68,44809 ms


Server results are better than embedded db's, although it should be the other way around. Many factors were different, less powerful server, classic .net vs .netcore, but perhaps Leveldb just works better on Linux.


Difference between embedded-db code and server-db code:
1. using LinqDb; and using LinqdbClient;
2. var db = new Db(path-to-folder); and var db = new Db(ip-address:port);