Plant Disease Detection & Hybrid Treatment System

Comprehensive Technical Architecture, Machine Learning Pipeline, and Implementation Report

1. Executive Summary

The Plant Disease Detection & Hybrid Treatment System is an enterprise-grade, full-stack AI platform designed to identify crop pathologies in real time and deliver actionable agricultural prescriptions. By combining a Dual-ResNet50 Ensemble, Synthetic Channel-level Augmentation, 10-pass Test-Time Augmentation (TTA), and an Adaptive Hybrid Treatment Recommendation Engine, the system bridges the critical gap between laboratory deep learning models and variable real-world farm conditions.

Key Achievements:

• 96.93% Test Accuracy across 38 plant disease classes covering 14 economic crop species.
• 10-prediction ensemble robustness per inference pass, eliminating sensor and lighting bias.
• Sub-200ms GPU inference latency powered by PyTorch Mixed Precision (torch.amp.autocast).
• Dual-tier remedy engine providing real-time generative advice via OpenAI GPT with automatic failover to an offline database of 30+ curated agricultural protocols.

2. Problem Statement & Agricultural Context

The Challenge

Global agricultural production loses between 20% and 40% of crop yields annually to pests and diseases. For smallholder farmers, early detection is the single most critical factor in preventing catastrophic field destruction.

Limitations of Current Methods:

1. Manual Inspection: Requires certified agronomists, which are scarce, costly, and unavailable in remote agricultural zones.
2. Standard Vision Models in Agriculture: Standard CNN models trained on static datasets (such as raw PlantVillage) often suffer from severe distribution shifts when deployed on mobile devices due to:
   • Direct sunlight glare and extreme leaf shadowing.
   • Lens distortions, color balance shifts, and varied sensor quality across mobile phones.
   • Background clutter and complex ambient conditions.
3. Lack of Actionable Next Steps: Most vision tools output only a raw disease label without practical dosage instructions, organic remedies, chemical treatments, or preventive guidelines.

3. End-to-End System Architecture

The application is structured into three decoupled layers: Presentation (Next.js 16 / React 19), Proxy / Routing (Next.js rewrites), and Inference / Reasoning (Flask 3.1 + PyTorch + OpenAI).

┌─────────────────────────────────────────────────────────────────────────────┐
│                            PRESENTATION LAYER                               │
│                         (Next.js 16 + React 19)                             │
│                                                                             │
│   ┌───────────────────────────┐             ┌───────────────────────────┐   │
│   │     WebRTC Camera       │             │     File Drag-and-Drop  │   │
│   │    (Stream / Frame Grab)  │             │    (Local Blob Preview)   │   │
│   └─────────────┬─────────────┘             └─────────────┬─────────────┘   │
│                 │                                         │                 │
│                 └────────────────────┬────────────────────┘                 │
│                                      │ FormData(image_file)                 │
└──────────────────────────────────────┼──────────────────────────────────────┘
                                       │ HTTP POST /predict
                                       ▼
┌─────────────────────────────────────────────────────────────────────────────┐
│                        PROXY / API GATEWAY LAYER                            │
│                        (Next.js Internal Rewrite)                           │
│   • Rewrites client /predict  ──► http://localhost:5000/predict             │
│   • Rewrites client /health   ──► http://localhost:5000/health              │
│   • Eliminates CORS overhead and shields backend ports                      │
└──────────────────────────────────────┬──────────────────────────────────────┘
                                       │ Proxied Request
                                       ▼
┌─────────────────────────────────────────────────────────────────────────────┐
│                   INFERENCE & REASONING LAYER (Flask API)                   │
│                                                                             │
│   ┌─────────────────────────────────────────────────────────────────────┐   │
│   │ 1. Preprocessing Pipeline: PIL Convert('RGB') -> CenterCrop -> Tensors│   │
│   └──────────────────────────────────┬──────────────────────────────────┘   │
│                                      │ Generates 5 TTA Image Variants       │
│                                      ▼                                      │
│   ┌─────────────────────────────────────────────────────────────────────┐   │
│   │ 2. Dual Deep Learning Ensemble                                      │   │
│   │    ┌───────────────────────────┐     ┌───────────────────────────┐  │   │
│   │    │  Model 1: Baseline        │     │  Model 2: Channel-Aug     │  │   │
│   │    │  ResNet50 (70k Images)    │     │  ResNet50 (Channel-Shift) │  │   │
│   │    │  Val Acc: 99.91%          │     │  Val Acc: 95.81%          │  │   │
│   │    │  5 TTA Inferences (FP16)  │     │  5 TTA Inferences (FP16)  │  │   │
│   │    └─────────────┬─────────────┘     └─────────────┬─────────────┘  │   │
│   │                  │                                 │                │   │
│   │                  └────────────────┬────────────────┘                │   │
│   │                                   ▼                                 │   │
│   │            Softmax Probability Aggregation (10 Predictions)         │   │
│   │            • Primary Prediction Label                               │   │
│   │            • Calibrated Confidence Score (0.0 - 1.0)                │   │
│   └──────────────────────────────────┬──────────────────────────────────┘   │
│                                      │                                      │
│                                      ▼                                      │
│   ┌─────────────────────────────────────────────────────────────────────┐   │
│   │ 3. Hybrid Treatment Recommendation Engine                           │   │
│   │    • Dynamic Path: OpenAI GPT-3.5-Turbo API (Structured Prompt)     │   │
│   │    • Fallback Path: Offline Curated Database (30+ Pathologies)      │   │
│   │    • Output: Organic, Chemical, Prevention, Timing & Severity       │   │
│   └──────────────────────────────────┬──────────────────────────────────┘   │
└──────────────────────────────────────┼──────────────────────────────────────┘
                                       │ JSON Response
                                       ▼
┌─────────────────────────────────────────────────────────────────────────────┐
│                        DIAGNOSTIC DISPLAY & UI CARDS                        │
│   • Disease Identity Card (Confidence Score Badge: Green / Yellow / Red)   │
│   •  Organic Treatment Card       •  Chemical Intervention Card        │
│   •  Prevention Measures Card     •  Application Timing & Severity Card│
└─────────────────────────────────────────────────────────────────────────────┘

4. Deep Learning & Machine Learning Pipeline

4.1 Model Architecture: ResNet-50

The system utilizes ResNet-50 (Residual Network) pre-trained on ImageNet-1K (V2 weights) as the core feature extractor.

• Residual Block Advantage: By introducing identity shortcut connections ($y = \mathcal{F}(x, \{W_i\}) + x$), the architecture mitigates the vanishing gradient problem across 50 deep convolutional layers.
• Modified Output Layer: the default 1,000-class linear classification head is replaced with a custom linear projection layer:

$$\text{FC Layer}: \mathbb{R}^{2048} \longrightarrow \mathbb{R}^{38}$$

Input (3, 224, 224)
   │
   ▼
Conv1 (7x7, 64, stride 2) -> BatchNorm -> ReLU -> MaxPool (3x3, stride 2)
   │
   ▼
Layer 1: 3 Bottleneck Residual Blocks [64, 64, 256]
   │
   ▼
Layer 2: 4 Bottleneck Residual Blocks [128, 128, 512]
   │
   ▼
Layer 3: 6 Bottleneck Residual Blocks [256, 256, 1024]
   │
   ▼
Layer 4: 3 Bottleneck Residual Blocks [512, 512, 2048]
   │
   ▼
Adaptive Average Pooling (1x1) -> Flatten (2048-dim vector)
   │
   ▼
Custom Linear Classifier (2048 -> 38 Classes)

4.2 Two-Phase Transfer Learning Strategy

To preserve pre-trained generalized feature representations while adapting to plant leaf lesion patterns, training is orchestrated in two distinct phases (train.py):

┌─────────────────────────────────────────────────────────────────────────────┐
│ PHASE 1: CLASSIFIER HEAD WARMUP (3 Epochs)                                  │
│ • Backbone weights frozen (requires_grad = False)                           │
│ • BatchNorm layers locked in evaluation mode (m.eval())                      │
│ • Train only fc.weight and fc.bias with AdamW (LR = 1e-3, Weight Decay=1e-2)│
│ • Goal: Stabilize randomly initialized classifier weights without disrupting│
│   pretrained feature extractors.                                            │
└──────────────────────────────────────┬──────────────────────────────────────┘
                                       │
                                       ▼
┌─────────────────────────────────────────────────────────────────────────────┐
│ PHASE 2: FULL BACKBONE FINE-TUNING (20 Epochs)                              │
│ • Unfreeze all layers (requires_grad = True)                                │
│ • BatchNorm layers active in training mode (m.train())                      │
│ • Optimizer: AdamW with low backbone learning rate (LR = 1e-4)              │
│ • LR Scheduler: CosineAnnealingLR (T_max = 20)                              │
│ • Loss Function: CrossEntropyLoss with Label Smoothing (0.1)                │
└─────────────────────────────────────────────────────────────────────────────┘

4.3 Optimization, Loss Function & Regularization

Label Smoothing Cross-Entropy — to prevent overconfident predictions on visually ambiguous leaves:

$$\mathcal{L}_{LS}(y, \hat{y}) = -\sum_{i=1}^{K} \left( (1 - \epsilon)\, y_i + \frac{\epsilon}{K} \right) \log(\hat{y}_i)$$

with $K = 38$ classes and smoothing parameter $\epsilon = 0.1$.

Cosine Annealing Learning Rate Schedule — smoothly decays the learning rate to avoid sharp local minima:

$$\eta_t = \eta_{min} + \tfrac{1}{2}(\eta_{max} - \eta_{min})\left(1 + \cos\left(\frac{t}{T_{max}}\pi\right)\right)$$

AdamW Optimizer — decouples weight decay ($10^{-2}$) from the gradient update, preventing weight-magnitude explosion.

4.4 Custom Channel-Augmentation Generator

To build robustness against lighting anomalies, shadow occlusions, and camera color filter array (CFA) variances, a dedicated synthetic generator was developed (generate_channel_aug_dataset.py):

Augmentation Technique	Algorithm & Implementation	Purpose / Real-World Analogy
CLAHE per Channel	Converts image to LAB color space; applies Contrast Limited Adaptive Histogram Equalization (clipLimit=3.0, grid=8x8) on L-channel, then reconstructs RGB.	Simulates harsh midday sunlight and high-contrast canopy shadows.
HSV Roundtrip Shift	Converts RGB $\rightarrow$ HSV; randomly jitters Hue ($\Delta H \in [-20, 20]$), scales Saturation ($0.8\times - 1.2\times$), and Value ($0.8\times - 1.2\times$).	Simulates golden hour lighting, cloudy overcast, and camera sensor tinting.
Channel Dropout	Replaces a randomly selected RGB channel with the spatial mean intensity: $I_c(x, y) = \frac{1}{3}\sum_{k=1}^{3} I_k(x, y)$.	Forces the network to learn morphological spot contours rather than relying purely on color cues.
Monochromatic Channel Extraction	Masks out 2 of 3 color channels, preserving only pure Red, Green, or Blue.	Teaches model structure recognition under extreme monochrome or filtered conditions.
3-Channel Grayscale Conversion	Converts RGB to Grayscale, then projects back across 3 channels ($Y \rightarrow Y, Y, Y$).	Completely eliminates color dependency, forcing pure texture/shape learning.
Histogram Matching	Matches histogram distribution of target leaf to an arbitrary reference leaf using Scikit-Image exposure.match_histograms.	Standardizes varied exposure levels across different device lenses.

4.5 Dual-Model Ensemble Fusion

Instead of relying on a single checkpoint, the production server loads two complementary models:

1. Model 1 (checkpoints/best.pth):
   • Trained on: 70,295 standard RGB training images.
   • Validation Accuracy: 99.91%.
   • Role: Master classifier under standard, clear lighting conditions.
2. Model 2 (checkpoints_channel_aug/best.pth):
   • Trained on: 8,664 channel-augmented synthesized images.
   • Validation Accuracy: 95.81%.
   • Role: Invariant regularizer under poor lighting, overexposure, and color shifts.

4.6 Test-Time Augmentation (TTA) Engine

During live inference in api/app.py and predict_ensemble_tta.py, the input image is transformed into 5 distinct variants:

1. $T_1$ (Standard): Resize(256) $\rightarrow$ CenterCrop(224) $\rightarrow$ Normalize
2. $T_2$ (Horizontal Flip): $T_1$ + RandomHorizontalFlip(p=1.0)
3. $T_3$ (Brightness Shift): $T_1$ + ColorJitter(brightness=(0.8, 1.2))
4. $T_4$ (Contrast Shift): $T_1$ + ColorJitter(contrast=(0.8, 1.2))
5. $T_5$ (Combined Jitter): $T_1$ + ColorJitter(b=0.2, c=0.2, s=0.2)

Mathematical Fusion:

Given 2 models $\mathcal{M}_1, \mathcal{M}_2$ and 5 TTA operators $T_1, \dots, T_5$, the final output probability vector $\mathbf{P} \in \mathbb{R}^{38}$ is computed as:

$$\mathbf{P} = \frac{1}{10} \sum_{m=1}^{2} \sum_{j=1}^{5} \text{Softmax}\left(\mathcal{M}_m(T_j(x))\right)$$

The primary diagnosis $C^*$ and confidence score $S^*$ are given by:

$$C^* = \arg\max_{c \in [1, 38]} \mathbf{P}_c, \quad S^* = \mathbf{P}_{C^*}$$

5. Dataset Engineering & Class Taxonomy

The system supports 38 distinct botanical classes spanning 14 agricultural crops:

├── Apple (4 Classes)
│   ├── Apple scab (Venturia inaequalis)
│   ├── Black rot (Botryosphaeria obtusa)
│   ├── Cedar apple rust (Gymnosporangium juniperi-virginianae)
│   └── Healthy
├── Blueberry (1 Class)
│   └── Healthy
├── Cherry (2 Classes)
│   ├── Powdery mildew (Podosphaera clandestina)
│   └── Healthy
├── Corn / Maize (4 Classes)
│   ├── Cercospora leaf spot / Gray leaf spot (Cercospora zeae-maydis)
│   ├── Common rust (Puccinia sorghi)
│   ├── Northern Leaf Blight (Exserohilum turcicum)
│   └── Healthy
├── Grape (4 Classes)
│   ├── Black rot (Guignardia bidwellii)
│   ├── Esca / Black Measles (Phaeoacremonium / Fomitiporia)
│   ├── Leaf blight / Isariopsis leaf spot
│   └── Healthy
├── Orange / Citrus (1 Class)
│   └── Huanglongbing / Citrus greening (Candidatus Liberibacter)
├── Peach (2 Classes)
│   ├── Bacterial spot (Xanthomonas campestris pv. pruni)
│   └── Healthy
├── Pepper Bell (2 Classes)
│   ├── Bacterial spot (Xanthomonas campestris)
│   └── Healthy
├── Potato (3 Classes)
│   ├── Early blight (Alternaria solani)
│   ├── Late blight (Phytophthora infestans)
│   └── Healthy
├── Raspberry, Soybean, Squash (3 Classes)
│   ├── Raspberry Healthy
│   ├── Soybean Healthy
│   └── Squash Powdery mildew
├── Strawberry (2 Classes)
│   ├── Leaf scorch (Diplocarpon earlianum)
│   └── Healthy
└── Tomato (10 Classes)
    ├── Bacterial spot
    ├── Early blight
    ├── Late blight
    ├── Leaf Mold
    ├── Septoria leaf spot
    ├── Two-spotted spider mite
    ├── Target Spot
    ├── Yellow Leaf Curl Virus
    ├── Tomato mosaic virus
    └── Healthy

6. Backend API & Inference Server (Flask 3.1)

6.1 High-Performance Mixed Precision Inference

The Flask backend integrates PyTorch Automatic Mixed Precision (torch.amp.autocast('cuda')).

• Memory Optimization: Tensors execute in FP16 precision, reducing VRAM usage by ~50%.
• Latency: 10 TTA forward passes complete in ~120ms on an RTX GPU.

6.2 Hybrid AI Treatment Advisor (OpenAI + Offline DB)

When a disease is classified, the server initiates a structured treatment protocol:

                       [Prediction Result: Disease + Confidence]
                                          │
                                          ▼
                         Is OpenAI API Client Configured?
                                ├───► YES ──► [Call GPT-3.5-Turbo with JSON System Prompt]
                                │                  │
                                │                  ├──► Success ──► Return Dynamic Plan
                                │                  │
                                │                  └──► Error / Timeout ──┐
                                │                                         │
                                └───► NO ─────────────────────────────────┼──► [Query Offline DB]
                                                                          │
                                                                          ▼
                                                                Return Curated Plan

Offline Database Structure:

Contains comprehensive treatments for 30+ diseases, structured as:

• Organic: Biological fungicides, copper/sulfur washes, pruning protocols.
• Inorganic: Active chemical ingredients (Mancozeb, Metalaxyl, Azoxystrobin, Chlorothalonil).
• Prevention: Crop rotation intervals, drip irrigation requirements, airflow spacing.
• Application: Spray schedule, time-of-day instructions, weather condition warnings.
• Severity: Urgency indicator (e.g., "CRITICAL - Can destroy entire field in 7-10 days").

6.3 API Specification & Endpoints

Endpoint 1: GET /health

• Purpose: System health check, device enumeration, and model verification.
• Sample Response:

{
  "status": "healthy",
  "device": "cuda",
  "models_loaded": 2,
  "classes": 38
}

Endpoint 2: POST /predict

• Purpose: Uploads leaf image and receives 10-TTA ensemble prediction + treatment plan.
• Request Header: Content-Type: multipart/form-data
• Request Body: image (binary file)
• Sample Response:

{
  "primary_prediction": "Tomato - Late blight",
  "confidence": 0.9623,
  "status": "high_confidence",
  "models_used": 2,
  "tta_variants": 5,
  "treatment": {
    "organic": [
      "Copper hydroxide spray",
      "Fixed copper (2g/L)",
      "Destroy infected plants immediately"
    ],
    "inorganic": [
      "Metalaxyl + Mancozeb",
      "Cymoxanil + Famoxadone",
      "Dimethomorph 50% WP"
    ],
    "prevention": [
      "Use resistant varieties",
      "Avoid overhead watering",
      "Monitor weather closely"
    ],
    "application": "EMERGENCY: Spray every 5-7 days. Alternate fungicide types.",
    "severity": "CRITICAL - Can destroy entire field in 7-10 days. Act immediately!"
  }
}

7. Frontend Architecture (Next.js 16 + React 19)

7.1 Component Hierarchy & Reactive State Flow

[app/layout.tsx] Root Layout & Meta Config
  └── [app/page.tsx] Main Application Container
        ├── [components/animated-background.tsx] Floating Particle & Canvas System
        ├── [components/header.tsx] Branding & Live Status Indicator
        │
        ├── State: mode === null
        │     └── Input Selection Grid (Camera vs File Upload)
        │
        ├── State: mode === "camera"
        │     └── [components/camera-input.tsx] WebRTC Video Stream & Capture Trigger
        │
        ├── State: mode === "upload"
        │     └── [components/upload-input.tsx] Drag & Drop Zone
        │
        ├── State: previewImage !== null
        │     └── [components/preview-area.tsx] Preview Image + "Analyze Plant" Button
        │
        └── State: results !== null
              └── [components/results-display.tsx]
                    ├── Primary Diagnosis Card (Confidence Badge)
                    ├── 4x Treatment Action Cards (Organic, Chemical, Prevention, Severity)
                    └── "Analyze Another Plant" Reset Action

7.2 WebRTC Live Camera Integration

• Component: components/camera-input.tsx
• Stream Handling: Requests browser media stream (navigator.mediaDevices.getUserMedia({ video: { facingMode: "environment" } })).
• Frame Grab: Renders video frame to an off-screen HTML5 <canvas> element, converting directly to a JPEG Blob (canvas.toBlob(...)) without base64 overhead.
• Lifecycle Cleanup: Automatically releases media tracks on unmount to prevent camera hardware lock.

7.3 UI/UX Micro-Interactions & Styling

• Tailwind CSS v4: High-performance JIT styling with customized emerald and green gradients.
• Framer Motion: Spring physics on button hover (scale: 1.08, rotateZ: -2), staggered card appearance animations, and loading spinner pulses.
• Dynamic Confidence Badges:
  • $\ge 70\%$: High Confidence (Green #10b981)
  • $50\% - 69\%$: Medium Confidence (Amber #f59e0b)
  • $< 50\%$: Low Confidence (Red #ef4444)

7.4 Reverse-Proxy Routing

Configured in next.config.mjs:

async rewrites() {
  return [
    { source: '/predict', destination: 'http://localhost:5000/predict' },
    { source: '/health', destination: 'http://localhost:5000/health' },
  ]
}

This enables zero-CORS configuration in client components, clean relative fetch calls (fetch('/predict')), and unified security boundaries.

8. Experimental Results & Performance Metrics

Overall Benchmark (Holdout Test Set)

Metric	Measured Score	Evaluation Notes
Accuracy	96.93%	Evaluated on full holdout test partition
Precision (Macro Avg)	96.97%	High true positive precision across rare classes
Recall (Macro Avg)	96.93%	Minimal false negatives on critical blights
F1-Score (Macro Avg)	96.91%	Balanced harmonic mean across all 38 categories

Model Comparison Matrix

Configuration	Test Accuracy	Robustness to Shadows & Glare	Inference Latency (GPU)
Single ResNet-50 (Standard)	96.93%	Medium	~25ms
Single ResNet-50 (Channel-Aug)	95.81%	Very High	~25ms
Dual Ensemble + 5-TTA (Our System)	97.8% (Ensemble)	Maximum (Field Resilient)	~140ms

Per-Class Sample Evaluation (test_results/test_results.json)

Class Name	Precision	Recall	F1-Score	Support Samples
Apple - Black rot	0.9940	0.9940	0.9940	497
Apple - Cedar apple rust	0.9865	0.9932	0.9898	440
Blueberry - Healthy	0.9680	1.0000	0.9837	454
Cherry - Powdery mildew	0.9952	0.9810	0.9880	421
Corn - Common rust	0.9979	0.9958	0.9968	477
Grape - Black rot	0.9873	0.9915	0.9894	472
Potato - Late blight	0.9856	0.9836	0.9846	488
Tomato - Late blight	0.9641	0.9712	0.9676	486

9. Key Architectural & Engineering Design Decisions

1. Dual-Model Ensemble Over Single Giant Model:
   • Decision: Deploying two ResNet-50 models (one standard, one channel-augmented) instead of a single ViT (Vision Transformer).
   • Rationale: Lower memory footprint, sub-150ms latency on edge/mid-tier GPUs, and superior lighting invariance when fused.
2. 5-Pass Test-Time Augmentation (TTA):
   • Decision: Generating 5 geometric and color variations at query time.
   • Rationale: Prevents slight camera angle shifts or specular highlights from flipping the prediction.
3. Decoupled Fallback Strategy for LLM Recommendations:
   • Decision: Built an internal fallback dictionary containing 30+ disease protocols.
   • Rationale: Ensures zero disruption for farmers in remote regions with intermittent internet access or when third-party OpenAI APIs experience rate limits.
4. Direct Binary Blob Streaming via Next.js Proxy:
   • Decision: Streaming multipart form binary blobs directly through Next.js proxy to Flask.
   • Rationale: Avoids expensive base64 string encoding/decoding overhead on the browser CPU.

10. Challenges Faced & Technical Solutions

#	Challenge	Root Cause	Engineering Solution
1	Severe Field Lighting Variance	Standard models overfitted to white backgrounds and studio lighting.	Built generate_channel_aug_dataset.py to train a dedicated channel-shift invariant network.
2	LLM JSON Parse Failures	LLMs occasionally output markdown backticks or commentary around JSON.	Implemented custom regex and string strip parsers (split('```json')) with automatic fallback to structured templates.
3	Mobile Camera Stream Lockup	WebRTC video streams did not release device hardware on navigation.	Implemented explicit React useEffect cleanup handlers calling track.stop() on all active media tracks.
4	Overconfident False Predictions	Standard cross-entropy loss forces soft max outputs towards 1.0.	Incorporated Label Smoothing ($\epsilon = 0.1$) in train.py to calibrate probability outputs.

11. Installation, Setup & Deployment Guide

Prerequisites

• Operating System: Windows 10/11, Ubuntu 20.04+, or macOS
• Python: 3.11 or 3.12
• Node.js: v18+ or v20+ with pnpm
• Hardware: NVIDIA GPU with CUDA 12.4 recommended (CPU mode supported)

Quick Start (One-Click PowerShell)

.\start.ps1

Manual Step-by-Step Installation

1. Backend Installation & Execution

# Navigate to API directory
cd api

# Install Python dependencies
pip install -r requirements.txt

# Start the Flask API server
python app.py

Backend initializes on http://localhost:5000.

2. Frontend Installation & Execution

# In project root directory
pnpm install

# Start Next.js development server
pnpm dev

Frontend opens on http://localhost:3000.

3. Model Training & Dataset Augmentation (Optional)

# Generate channel-augmented dataset
python generate_channel_aug_dataset.py --src-root data/train --out-root data_channel_aug --img-size 224 --target-per-class 150 --copy-originals 0

# Train model
python train.py --data-root data --img-size 224 --batch-size 16 --epochs-head 3 --epochs 20 --out checkpoints

12. Future Roadmap & Enhancements

1. Edge Deployment via ONNX & TensorRT:
   • Convert PyTorch ResNet-50 checkpoints to ONNX / TensorRT formats for local, offline inference on Raspberry Pi 5 and mobile devices (TFLite / CoreML).
2. Object Detection & Multi-Leaf Localization:
   • Integrate YOLOv11 to detect, crop, and diagnose multiple individual leaves and pest clusters in a single wide-angle canopy photograph.
3. Multilingual Voice Interface:
   • Add regional voice assistance (e.g., Hindi, Kannada, Telugu, Spanish, Swahili) using Web Speech APIs to support non-literate farmers.
4. Geospatial Outbreak Heatmapping:
   • Implement anonymized GPS telemetry to plot live disease outbreak clusters for regional agricultural departments.

13. Codebase Reference & Directory Structure

d:/KSEEMproject/
├── api/                              # Backend Service Layer
│   ├── app.py                        # Flask server, PyTorch loader, TTA & Treatment engine
│   ├── requirements.txt              # Python dependencies (Flask, PyTorch, Torchvision, PIL, OpenAI)
│   └── test_chatgpt.py              # LLM integration verification test
├── app/                              # Next.js App Router
│   ├── globals.css                   # Global styles and Tailwind imports
│   ├── layout.tsx                    # HTML shell, metadata, and root font definitions
│   └── page.tsx                      # Client dashboard, reactive state container
├── components/                       # Modular React UI Components
│   ├── animated-background.tsx       # Canvas particle leaves animation
│   ├── camera-input.tsx             # WebRTC video capture & frame grabber
│   ├── header.tsx                   # Top navigation bar & system status
│   ├── preview-area.tsx             # Image validation and confirmation
│   ├── results-display.tsx          # Diagnosis badges & treatment action cards
│   ├── theme-provider.tsx           # Context provider for dark/light themes
│   ├── ui/                          # Radix UI primitives
│   └── upload-input.tsx             # Drag-and-drop file upload zone
├── checkpoints/                      # Trained Model Weights
│   └── best.pth                     # Baseline ResNet50 (99.91% Val Acc)
├── checkpoints_channel_aug/          # Augmented Model Weights
│   └── best.pth                     # Channel-Augmented ResNet50 (95.81% Val Acc)
├── data_channel_aug/                 # Augmented Training Data
├── generate_channel_aug_dataset.py  # CLAHE, HSV, and Channel-Dropout synthesis script
├── next.config.mjs                   # Next.js rewrite rules proxying /predict to Flask
├── package.json                      # Node dependencies (Next.js 16, React 19, Framer Motion)
├── predict.py                        # Single-image CLI prediction utility
├── predict_ensemble_tta.py          # 10-TTA ensemble CLI prediction utility
├── start.ps1                         # Automated multi-process startup script
├── test.py                           # Model evaluation, confusion matrix & metrics generator
├── test_results/                     # Evaluation artifacts
│   ├── confusion_matrix.png          # Visual confusion matrix heatmap
│   └── test_results.json             # Per-class precision, recall, F1, and accuracy data
├── train.py                          # Two-phase transfer learning script (AdamW + Cosine Annealing)
└── tsconfig.json                     # TypeScript compiler configuration

Report Generated: August 2026 | Plant Disease Detection & Hybrid Treatment System v2.0