Benchmarks

Litdata has been benchmarked against a few other libraries. Here are the results and the associated Lightning Studio.

Streaming data

Framework	Images / sec 1st Epoch (float32)	Images / sec 2nd Epoch (float32)	Images / sec 1st Epoch (torch16)	Images / sec 2nd Epoch (torch16)
LitData	5800	6589	6282	7221
Web Dataset	3134	3924	3343	4424
Mosaic ML	2898	5099	2809	5158

Imagenet-1.2M dataset
To align with other benchmarks, we measured the streaming speed
To align with other benchmarks, we measured the streaming speed (images per second) loaded from AWS S3 for several frameworks.

LitData optimizes the Imagenet dataset for fast training 3-5x faster than other frameworks:

Time to optimize 1.2 million ImageNet images (Faster is better):

Framework	Train Conversion Time	Val Conversion Time	Dataset Size	# Files
LitData	10:05 min	00:30 min	143.1 GB	2.339
Web Dataset	32:36 min	01:22 min	147.8 GB	1.144
Mosaic ML	49:49 min	01:04 min	143.1 GB	2.298

Was this page helpful?