Knowledge Base | Abdullah Consulting

Configuration

Python — Log Anomaly Detection with scikit-learn

Automated log pattern analysis inspired by production support workflows.

Python — Log Anomaly Detection with scikit-learn

import re
from sklearn.feature_extraction.text import TfidfVectorizer
from sklearn.ensemble import IsolationForest
import pandas as pd

def parse_logs(log_file):
    patterns = []
    with open(log_file) as f:
        for line in f:
            # Normalize timestamps and IPs
            clean = re.sub(r'\d{4}-\d{2}-\d{2}[T ]\d{2}:\d{2}:\d{2}', '<TS>', line)
            clean = re.sub(r'\d+\.\d+\.\d+\.\d+', '<IP>', clean)
            patterns.append(clean.strip())
    return patterns

def detect_anomalies(logs, contamination=0.05):
    vectorizer = TfidfVectorizer(max_features=1000)
    X = vectorizer.fit_transform(logs)
    model = IsolationForest(contamination=contamination, random_state=42)
    predictions = model.fit_predict(X)
    anomalies = [logs[i] for i, p in enumerate(predictions) if p == -1]
    return anomalies

if __name__ == '__main__':
    logs = parse_logs('/var/log/messaging/app.log')
    for entry in detect_anomalies(logs):
        print(f'ANOMALY: {entry}')

Configuration

OpenAI API — Incident Summarization

Use LLMs to generate RCA drafts from Splunk log exports.

OpenAI API — Incident Summarization

import openai
import os

openai.api_key = os.environ['OPENAI_API_KEY']

def summarize_incident(log_snippet, alert_context):
    prompt = f"""You are a senior SRE. Analyze this incident data and produce:
1. Brief summary (2 sentences)
2. Likely root cause
3. Recommended next steps

Alert: {alert_context}

Logs:
{log_snippet}
"""
    response = openai.chat.completions.create(
        model='gpt-4o-mini',
        messages=[{'role': 'user', 'content': prompt}],
        temperature=0.2,
        max_tokens=500
    )
    return response.choices[0].message.content

Knowledge Base

MLOps Checklist

Best practices for deploying AI models in production.

MLOps Checklist

## Model Deployment Checklist

- [ ] Version control for training data and model artifacts
- [ ] Automated retraining pipeline with drift detection
- [ ] A/B testing framework for model rollouts
- [ ] Latency SLA monitoring (< 200ms for inference)
- [ ] Fallback to rule-based logic when model confidence is low
- [ ] Audit logging for all AI-generated recommendations
- [ ] Cost monitoring for API-based LLM usage

## AWS AI Services Quick Reference

| Service | Use Case |
|---------|----------|
| SageMaker | Custom ML model training & hosting |
| Bedrock | Managed LLM access (Claude, Titan) |
| Comprehend | NLP — sentiment, entity extraction |
| Forecast | Time-series demand prediction |