Technologies/Meilisearch/meilisearch.tasks.processing

MeilisearchMetric

meilisearch.tasks.processing

Tasks currently processing

Dimensions:None

Available on:

Prometheus (1)Native (1)

Interface Metrics (2)

Prometheus

meilisearch_tasks_processing

Current number of tasks being processed

Dimensions:None

Native

tasks.processing

Number of tasks in processing status

Dimensions:None

Sources

meilisearch_tasks_processingdocs.meilisearch.com

tasks.processingwww.meilisearch.com

Technical Annotations (35)

Configuration Parameters (10)

MEILI_EXPERIMENTAL_MAX_NUMBER_OF_BATCHED_TASKSrecommended: 5-50

Limits concurrent task processing; lower values (5-20) for large batches, higher (up to 50) for smaller batches

MEILI_MAX_INDEXING_THREADSrecommended: 20

Example thread limit for 48-core systems during large imports

max_payload_sizerecommended: 100MB (default, can be increased)

Maximum size for batch document uploads

MEILI_MAX_INDEXING_MEMORYrecommended: depends on RAM

Set to control memory budget, but may not be fully respected with multiple threads

--experimental-reduce-indexing-memory-usagerecommended: enabled

Significantly reduces memory consumption during indexing operations

--max-indexing-threadsrecommended: 1

Reducing thread count can help constrain memory usage

--log-levelrecommended: DEBUG

Enables logging of memory allocation metrics at indexing start

MAX_MAP_COUNTrecommended: 262144

System-level memory-mapped file limit for handling bulk document operations

NOFILE_SOFTrecommended: 65536

Soft file descriptor limit required for high-volume document indexing

NOFILE_HARDrecommended: 65536

Hard file descriptor limit required for high-volume document indexing

Error Signatures (9)

io_errorerror code

I/O error (os error 5)error code

408http status

Request Timeouthttp status

Expected response code 200, got 408log pattern

Error parsing response JSON: invalid type: null, expected struct MeilisearchErrorexception

OOM killedexit code

meilisearch.service: Failed with result 'oom-kill'log pattern

Main process exited, code=killed, status=9/KILLexit code

CLI Commands (3)

docker statsmonitoring

--experimental-reduce-indexing-memory-usageremediation

--log-level DEBUGdiagnostic

Technical References (13)

autobatchingconceptactix-webcomponenttokiocomponentgrenad sortercomponentmillicomponenttask schedulercomponentauto-batchingconceptattributesconceptIndexStatus::BeingResizedcomponentprepare_for_closingconceptBBQueuecomponentextractor memorycomponentbatch indexingconcept

Related Insights (12)

Unlimited task autobatching triggers resource exhaustion and I/O failurescritical

▸

HTTP 408 timeout errors during concurrent indexing and querieswarning

▸

Memory pressure with large datasets and parallel index operationswarning

▸

Slow bulk document indexationwarning

▸

OOM crashes from auto-batching too many taskscritical

▸

Index update backlog under high-frequency single-document writeswarning

▸

Index unavailable during resize operationinfo

▸

Multiple indexing threads multiply memory consumption regardless of limitswarning

▸

Concurrent indexing operations amplify memory consumptionwarning

▸

Meilisearch OOM kill during indexing operationscritical

▸

Bulk document addition causes complete API freezecritical

▸

Large batch document indexing monopolizes workers causing HTTP 408warning

▸