Slow performance while having large array of facts.

Open
#324 10 comments 6 reactions 0 assignees View on GitHub

Nobody has claimed this yet.

Assessment

Difficulty
4/5
Estimated time
3-5 days
Newbie friendliness
42/100
Issue type
Bug
Clarity
Mostly clear
Activity status
Quiet
Tech stack
javascript
Domain
backend, performance

Research direction

No repository file or test is named. Start by reproducing the supplied benchmark around Engine.run with 80,000 entries and compare it with the plain JavaScript lookup; done should be a measured, actionable performance improvement or a clearly documented limitation.

Written by the indexing model from the issue text.

Description

I have integrated json-rules-engine with a project I am working on and the performance seems much slower than I would expect. I'm using the package to do a simple lookup at another set of facts.

const lookupFacts = [
    {  col1: '', col2: '', col3: ''},
    {  col1: '', col2: '', col3: ''},
    {  col1: '', col2: '', col3: ''}
]

const filterRule = {
  conditions: {
    all: [
      {
        path: "$.col1",
        fact: "fact",
        value: {
          path: "$.col1",
          fact: "lookup"
        },
        operator: "equal"
      },
      {
        path: "$.col2",
        fact: "data",
        value: {
          path: "$.col2",
          fact: "lookup"
        },
        operator: "equal"
      },
      {
        path: "$.col3",
        fact: "data",
        value: {
          path: "$.col3",
          fact: "lookup"
        },
        operator: "equal"
      }
    ],
    event: {
      type: 'filter-event'
    }
  }
}

const engine = new require('json-rules-engine').Engine()

let filteredMatchs = await Promise.all(lookupFacts.map((lookup) => {
  return engine.run({ lookup, data })
    .then(({ events }) => events.length > 0 ? lookup : false)
    .catch((err) => false)
    }))
    .then((values) => values.filter((value) => value))

In my case, the lookupfacts array seems to contain about 80000 entries and it takes around 30000 ms to complete. Whereas doing the same comparison using simple javascript code takes about 10-15 ms only.

I will only be not be having any dynamic data in the flow. Is there a way to improve performance?

Thanks

Dominant language
JavaScript
Stars
3.1k
Forks
507
No merged PRs in 30d

Contributor guide

No contributing guide indexed for this repository

First steps

  1. Read the whole issue, then the project's contributing guide.
  2. Comment on the issue to say you are picking it up — it saves two people doing the same work.
  3. Fork the repository and make your change on a branch.
  4. Open a pull request that references the issue number.

More from CacheControl/json-rules-engine

All issues in CacheControl/json-rules-engine

Similar issues

More JavaScript issues

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.