Theunet AI
Current AI ResearchResearch in progress

Building an Ethical, Traceable AI System for Kenyan GBV News

An ongoing master’s research project combining multilingual news classification, location extraction, reproducible data engineering and privacy-aware human review.

4 minute read

Business or research question

The question behind the system

How can multilingual Kenyan news be classified and mapped quickly while preserving data quality, privacy, traceability and human oversight?

Plain-language overview

How the system works

This ongoing Master of Science in Artificial Intelligence research begins with reproducible collection rather than model training. It discovers Kenyan digital-news material through web archives, preserves original HTML as evidence and writes normalised, versioned JSON for analysis.

Content hashes, parser versions and article identities connect each processed record to its source evidence. Google Cloud Storage holds raw, processed and run artefacts, while a PostgreSQL-based metadata layer tracks collection runs, article versions and lineage.

The current system remains in a deterministic data-collection phase. Collection infrastructure, custom extraction parsers, identity controls and a read-only Flask monitor are implemented. Human annotation, multilingual classification, location extraction, geocoding and privacy-aware mapping remain work in progress or planned capabilities.

Project stage

Completed, current and planned work

Work in progress

  • L0 extraction-quality review
  • L1 Kenya-relevance review
  • L2 GBV-relevance review
  • Reproducible human-label storage linked to article versions

Planned capabilities

  • L3 GBV-type enrichment
  • L4 location extraction and geocoding
  • L5 privacy-risk review
  • Multilingual model evaluation and privacy-aware analytical mapping

Completed infrastructure

  • Cloud storage and PostgreSQL-based metadata integration
  • Archive discovery and custom extraction parsers
  • Article identity, content hashing and versioned processing
  • Collection-run state and a read-only Flask monitor

Architecture

From source to usable output

The sequence below shows how information moves through the system.

  1. 01Digital news and archive sources
  2. 02Collector
  3. 03Raw HTML in cloud storage
  4. 04Parser and content hashing
  5. 05Versioned JSON in cloud storage
  6. 06Metadata and lineage database
  7. 07Human annotation gates
  8. 08Model classification and location extraction
  9. 09Privacy-aware analytical map

Technical deep dive

What happens inside the workflow

Raw evidence first

The collector writes original HTML to cloud storage before parsing. If later indexing fails, processing can resume from stored artefacts without requesting the source again.

Identity and versioning

A stable article identity represents the source and canonical URL. A separate content hash identifies the normalised payload. Together they prevent duplicate logical records while preserving changed extraction versions.

Layered annotation

L0 checks extraction quality. L1 labels Kenya relevance as Kenya, Not Kenya or Ambiguous. L2 labels GBV relevance as GBV, Not GBV or Borderline. Only after the corpus is stable do L3 GBV type, L4 location and L5 privacy risk add deeper information.

Important design decisions

Choices that shape reliability

01Preserve raw HTML before transformation so future parsers can reproduce the record.

02Use idempotent identities and hashes so repeated collection does not create duplicate logical articles.

03Check technical validity and relevance before deeper annotation or model work.

04Keep consequential labels linked to the exact article and extraction version reviewed.

The system checks whether an article is technically valid and relevant before asking annotators or models to extract deeper information. This avoids wasting effort on broken, unrelated or unsafe content.

Risks and trade-offs

Where careful judgement matters

  • Archive gaps can create non-random gaps in the dataset
  • Historical page layouts make parsers fragile
  • Publication dates and geographic relevance can be ambiguous
  • Sensitive reporting requires privacy checks before mapping
  • Human review is required before weak labels become research data

What organisations can achieve

Customer possibilities

These are possible applications of the approach, not reported outcomes from the source project.

  • Build traceable document and news-analysis pipelines
  • Preserve original evidence for later reprocessing
  • Monitor collection and processing workflows
  • Combine automated classification with human review
  • Extract geographic information from unstructured text
  • Introduce privacy checks before publishing analytical outputs
  • Maintain multiple versions without losing lineage

Technologies and capabilities

Technical themes

  • Python
  • Flask
  • Google Cloud Storage
  • PostgreSQL
  • Supabase
  • Data lineage
  • Content hashing
  • Idempotent ingestion
  • Human annotation
  • Multilingual transformers
  • Natural-language processing
  • Named-entity recognition
  • Geocoding
  • Responsible AI

Related insight

Turning Agricultural Research into Personalised Farmer Guidance

How structured farm inputs, research rules and analytical models can be combined to deliver practical recommendations through accessible digital channels.

Read the related insight

What could your organisation build from its data?

Tell us about the problem, available information and intended outcome.

Discuss Your Project