David Carlos: GSoC: The evolution of Kiskadee models.

The 0.1 release of Kiskadee [1] brought a minimal set of
functionalities that permitted us to monitor some targets and analysis some
packages, serving more as a proof of concept than as a real monitoring system.
One of the limitations of the 0.1 release was that Kiskadee was saving all the
analysis in a single field on the packages table. Every time that we need to
generate a new analysis for a same package version, we appended the result on
the analysis field. The next two code snippet was picked up from the Kiskadee
source code.

reports = []
...
with kiskadee.helpers.chdir(path):
        kiskadee.logger.debug('ANALYSIS: Unpacked!')
        analyzers = plugin.analyzers()
        for analyzer in analyzers:
                kiskadee.logger.debug('ANALYSIS: running %s ...' % analyzer)
                analysis = kiskadee.analyzers.run(analyzer, path)
                firehose_report = kiskadee.converter.to_firehose(analysis,
                                                         analyzer)
                reports.append(str(firehose_report))
kiskadee.logger.debug('ANALYSIS: DONE running %s' % analyzer)
return reports
all_analyses = 'n'.join(reports)

Note that we generate a analysis for each plugin analyzer, and append it in
the reports array. In other part of the runner code we generate, using a
join function, a single string with all the analysis made. A bad implementation
to a issue that should be solved by changing the Kiskadee models in order to
support several analysis for a same package version, and that was what we
have done with the PR #24.With
this pull request, Kiskadee will be able to save different analysis from a
same package version made by different analyzers. A lot of refactoring was made
in the Runner component, specially because was difficult to implement
new tests for it. The responsibility of save a analysis was removed from the
analyze method, and moved to the _save_source_analysis.

def _save_source_analysis(source_to_analysis, analysis, analyzer, session):
if analysis is None:
        return None

source_name = source_to_analysis['name']
source_version = source_to_analysis['version']

package = (
        session.query(kiskadee.model.Package)
        .filter(kiskadee.model.Package.name == source_name).first()
)
version_id = package.versions[-1].id
_analysis = kiskadee.model.Analysis()
try:
        _analyzer = session.query(kiskadee.model.Analyzer).
            filter(kiskadee.model.Analyzer.name == analyzer).first()
        _analysis.analyzer_id = _analyzer.id
        _analysis.version_id = version_id
        _analysis.raw = analysis
        session.add(_analysis)
except Exception as err:
        kiskadee.logger.debug(
        "The required analyzer was not registered on Kiskadee"
        )
        kiskadee.logger.debug(err)

With this pull request two new models was added to Kiskadee, analysis and
analyzers. The Diagram 1 presents the current database structure of
Kiskadee.

Diagram 1: Kiskadee database structure.

With the PR #24 we will fix
some other issues:

  • Docker not run properly on the Jenkins VM. Issue #23
    • Docker and selinux are not good friends so we have to change
      the way that we were creating a container volume to run the analyzers.
  • CI integration. Issue #7
    • We now have a Jenkins instance running our tests for every
      push made to the pagure repository. In the future we want to
      add a continuous deploy to every code that be merged on
      master branch.
[1]https://pagure.io/kiskadee


Source From: fedoraplanet.org.
Original article title: David Carlos: GSoC: The evolution of Kiskadee models..
This full article can be read at: David Carlos: GSoC: The evolution of Kiskadee models..

Advertisement


Random Article You May Like

Leave a Reply

Your email address will not be published. Required fields are marked *

*
*