The 0.1 release of Kiskadee  brought a minimal set of
functionalities that permitted us to monitor some targets and analysis some
packages, serving more as a proof of concept than as a real monitoring system.
One of the limitations of the 0.1 release was that Kiskadee was saving all the
analysis in a single field on the packages table. Every time that we need to
generate a new analysis for a same package version, we appended the result on
the analysis field. The next two code snippet was picked up from the Kiskadee
reports =  ... with kiskadee.helpers.chdir(path): kiskadee.logger.debug('ANALYSIS: Unpacked!') analyzers = plugin.analyzers() for analyzer in analyzers: kiskadee.logger.debug('ANALYSIS: running %s ...' % analyzer) analysis = kiskadee.analyzers.run(analyzer, path) firehose_report = kiskadee.converter.to_firehose(analysis, analyzer) reports.append(str(firehose_report)) kiskadee.logger.debug('ANALYSIS: DONE running %s' % analyzer) return reports
all_analyses = 'n'.join(reports)
Note that we generate a analysis for each plugin analyzer, and append it in
the reports array. In other part of the runner code we generate, using a
join function, a single string with all the analysis made. A bad implementation
to a issue that should be solved by changing the Kiskadee models in order to
support several analysis for a same package version, and that was what we
have done with the PR #24.With
this pull request, Kiskadee will be able to save different analysis from a
same package version made by different analyzers. A lot of refactoring was made
in the Runner component, specially because was difficult to implement
new tests for it. The responsibility of save a analysis was removed from the
analyze method, and moved to the _save_source_analysis.
def _save_source_analysis(source_to_analysis, analysis, analyzer, session): if analysis is None: return None source_name = source_to_analysis['name'] source_version = source_to_analysis['version'] package = ( session.query(kiskadee.model.Package) .filter(kiskadee.model.Package.name == source_name).first() ) version_id = package.versions[-1].id _analysis = kiskadee.model.Analysis() try: _analyzer = session.query(kiskadee.model.Analyzer). filter(kiskadee.model.Analyzer.name == analyzer).first() _analysis.analyzer_id = _analyzer.id _analysis.version_id = version_id _analysis.raw = analysis session.add(_analysis) except Exception as err: kiskadee.logger.debug( "The required analyzer was not registered on Kiskadee" ) kiskadee.logger.debug(err)
With this pull request two new models was added to Kiskadee, analysis and
analyzers. The Diagram 1 presents the current database structure of
With the PR #24 we will fix
some other issues:
- Docker not run properly on the Jenkins VM. Issue #23
- Docker and selinux are not good friends so we have to change
the way that we were creating a container volume to run the analyzers.
- CI integration. Issue #7
- We now have a Jenkins instance running our tests for every
push made to the pagure repository. In the future we want to
add a continuous deploy to every code that be merged on
Source From: fedoraplanet.org.
Original article title: David Carlos: GSoC: The evolution of Kiskadee models..
This full article can be read at: David Carlos: GSoC: The evolution of Kiskadee models..