Unable to build extension with profiling support

Open
#155 0 comments 1 reaction 0 assignees View on GitHub

Nobody has claimed this yet.

Assessment

Difficulty
4/5
Estimated time
3-5 days
Newbie friendliness
35/100
Issue type
Bug
Clarity
Mostly clear
Activity status
Stale
Tech stack
c, python
Domain
api, tooling

Research direction

Start with yara-python.c around the profiling_info() implementation and compare its PROFILING_ENABLED guards and clock_ticks/time_cost accesses with the current libyara profiling API. Reproduce the issue with python setup.py build --enable-profiling, then inspect how profiling information is stored in scanners. Done means profiling_info() works with the current libyara code without the reported build or runtime error.

Written by the indexing model from the issue text.

Description

I'm trying to build yara-python with profiling support.

The command python setup.py build --enable-profiling succeeds, but the resulting extension fails with "libyara compiled without profiling support" message when calling profiling_info() on a rules object.

yara-python.c seems to be outdated with respect to libyara in the profiling support.

First of all the preprocessor definition is PROFILING_ENABLED when it should be YR_PROFILING_ENABLED

However after changing that I got errors about clock_ticks not being member of YR_RULE and YR_STRING.

I saw in the code that clock_ticks was renamed as time_cost at some point, but later it was removed.

The only way I was able to compile the extension with the support for profiling was going back to version 3.11 in both yara-python and libyara, and also do some changes in the yara-python.c code (rename clock_ticks as time_cost and commenting out the addition of time_costs from YR_STRINGs since it seems to be member of YR_RULE only in this version).

So my modified yara-python.c at line 1619 looks like this:

#ifdef PROFILING_ENABLED
  PyObject* object;
  PyObject* result;

  YR_RULES* rules = ((Rules*) self)->rules;
  YR_RULE* rule;
  YR_STRING* string;

  char key[512];
  uint64_t time_cost;

  result = PyDict_New();

  yr_rules_foreach(rules, rule)
  {
    time_cost = rule->time_cost;

    /*yr_rule_strings_foreach(rule, string)
    {
      time_cost += string->time_cost;
    }*/

    snprintf(key, sizeof(key), "%s:%s", rule->ns->name, rule->identifier);

    object = PyLong_FromLongLong(time_cost);
    PyDict_SetItemString(result, key, object);
    Py_DECREF(object);
  }

  return result;
#else
  return PyErr_Format(YaraError, "libyara compiled without profiling support");
#endif

My first question is if I'm losing something because of not adding the YR_STRINGs time costs, since I don't know what they represented.

The second is if it would need too much rework in yara-python.c to support profiling with the latest libyara code.
Looking at the code, it seems that profiling information is now stored in "scanners", and I don't see any mention of scanners in yara-python.c.

Dominant language
C
Stars
754
Forks
190
PR merge metrics
No merged PRs in 30d

Contributor guide

No contributing guide indexed for this repository

First steps

  1. Read the whole issue, then the project's contributing guide.
  2. Comment on the issue to say you are picking it up — it saves two people doing the same work.
  3. Fork the repository and make your change on a branch.
  4. Open a pull request that references the issue number.

More from VirusTotal/yara-python

All issues in VirusTotal/yara-python

Similar issues

More C issues

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.