---
name: school-records-automation
title: School Records Automation
version: 1.0.0
description: Automate student data management from UDISE+ and similar government education portals. Extract, clean, organize, and maintain school records in Excel.
author: Hermes Agent for Piyush (Senior Clerk, Government Highschool)
last_updated: 2026-05-08
user_profile:
  name: Piyush
  role: Senior Clerk, Government Highschool (Gujarat)
  environment: Ubuntu desktop (NOT WSL)
  technical_level: Non-technical; prefers simple Hindi/Gujarati explanations
  primary_tasks: Excel management, student records, UDISE+ portal updates
  school_udise_code: 24110206411
  communication_style:
    - Uses simple, non-technical language
    - Prefers Hindi and Gujarati
    - Will ask for clarification when unsure
    - Values practical, step-by-step guidance
---

## 📋 Overview

This skill covers **automating student data workflows** for school administrative staff. When government portals like UDISE+ need regular data updates, this automation pattern handles:

- Portal login automation (Selenium)
- Data extraction (HTML scrap / file download)
- Excel organization (multiple sheets, clean structure)
- Data validation and cleaning
- Report generation

**Target user:** Non-technical school clerks/administrators who work with Excel daily but need to pull data from web portals.

---

## 🎯 When to Use This Skill

Use this skill when you need to:
- Download student data from UDISE+ or similar education department portals
- Convert downloaded data into organized Excel format with multiple sheets
- Maintain a master student database with regular updates
- Generate class-wise, gender-wise reports automatically
- Create individual student report cards

**Not for:** One-time simple data entry, manual copy-paste tasks.

---

## 🔧 Prerequisites

### 1. System Requirements
- **OS:** Ubuntu Linux (desktop, NOT WSL)
- **Python:** 3.11+ with uv package manager
- **Browser:** Google Chrome installed
- **Permissions:** User can run scripts from home directory

### 2. Python Environment Setup

```bash
# Create dedicated directory
mkdir -p ~/UDISE_Data
cd ~/UDISE_Data

# Create virtual environment
python3 -m venv venv
source venv/bin/activate

# Install required packages
pip install selenium pandas openpyxl webdriver-manager
```

### 3. Required Credentials
- UDISE+ portal username (usually school email)
- UDISE+ portal password
- School's UDISE code (11-digit number)

---

## 📁 Directory Structure

```
~/UDISE_Data/
├── venv/                          ← Python virtual environment
├── student_master_template.xlsx   ← Template with 4 sheets
├── UDISE_Student_Master.xlsx      ← Main database (updated regularly)
├── ReportCards/                   ← Individual student reports (HTML)
│   ├── report_2024001.html
│   └── ...
├── udise_student_manager.py       ← Main download & organize script
├── udise_data_processor.py        ← Clean & analyze data
└── excel_helper_functions.py      ← Reusable functions
```

---

## 🚀 Workflow Step-by-Step

### Phase 1: Initial Setup (One-time)

**Step 1:** Navigate to your directory
```bash
cd ~/UDISE_Data
source venv/bin/activate
```

**Step 2:** Place the three Python scripts:
- `udise_student_manager.py`
- `udise_data_processor.py`
- `excel_helper_functions.py`

**Step 3:** Create the master template (auto-created on first run)

---

### Phase 2: Data Download (Monthly/As Needed)

**Step 1:** Run the main manager
```bash
python3 ~/udise_student_manager.py
```

**Step 2:** Enter credentials when prompted:
```
Username: [your UDISE+ username]
Password: [your password]
School Code: [24110206411]
```

**What happens automatically:**
1. Chrome browser launches
2. Logs into UDISE+ portal
3. Navigates to student section
4. Downloads/scrapes student data
5. Saves to `~/UDISE_Data/`
6. Creates `UDISE_Student_Master.xlsx` with 4 sheets:
   - `Basic_Info` — personal details
   - `Academic` — class, section, roll number
   - `Attendance` — monthly attendance records
   - `Fees` — fee payment status

**Step 3:** Close browser when done

---

### Phase 3: Data Processing (After Download)

**Run the processor:**
```bash
python3 ~/udise_data_processor.py
```

**Automatically:**
- Cleans data (standardizes names, phone numbers, gender)
- Finds duplicates, missing values
- Generates summary statistics
- Creates class-wise sheets in the same Excel
- Saves as `Student_Data_Cleaned_[timestamp].xlsx`

---

### Phase 4: Manual Updates (In Excel)

Open `UDISE_Student_Master.xlsx` in LibreOffice Calc or Excel:

**Common tasks:**
1. **Add new student:** Append row in `Basic_Info` sheet
2. **Update class:** Edit `Class` column in `Academic` sheet
3. **Mark fees paid:** Update `Fees` sheet
4. **Fix phone numbers:** Use Find & Replace

**Save and backup:**
```bash
cp ~/UDISE_Data/UDISE_Student_Master.xlsx ~/UDISE_Data/Backup_$(date +%Y%m%d).xlsx
```

---

## 📊 Excel Structure Reference

### Sheet: Basic_Info
| Column | Type | Description |
|--------|------|-------------|
| UDISE_Code | text | 11-digit school code |
| Admission_No | text | Unique student ID |
| Name | text | Full name |
| Father_Name | text | Father's name |
| Mother_Name | text | Mother's name |
| DOB | date | Date of Birth |
| Gender | text | Male/Female/Other |
| Caste | text | SC/ST/OBC/General |
| Category | text | Social category |
| Aadhaar_No | text | 12-digit Aadhaar |
| Mobile_No | text | 10-digit phone |
| Email | text | Email address |
| Address | text | Residential address |

### Sheet: Academic
| Column | Type | Description |
|--------|------|-------------|
| Admission_No | text | Must match Basic_Info |
| Class | text | 1-12 |
| Section | text | A, B, C |
| Roll_No | integer | Roll number |
| Medium | text | Hindi/English/Gujarati |
| Year | text | 2024-25 |
| Previous_School | text | If transferred |
| Transfer_Certificate_No | text | TC number |

### Sheet: Attendance
| Column | Type | Description |
|--------|------|-------------|
| Admission_No | text | Student ID |
| Month | text | Jan, Feb... |
| Working_Days | integer | Total school days |
| Days_Present | integer | Days attended |
| Days_Absent | integer | Days absent |
| Attendance_Pct | float | % attendance |

### Sheet: Fees
| Column | Type | Description |
|--------|------|-------------|
| Admission_No | text | Student ID |
| Fee_Type | text | Tuition, Exam, Development |
| Amount | float | Total amount |
| Paid | float | Amount paid |
| Due | float | Balance due |
| Last_Payment_Date | date | Last payment date |

---

## 🔐 Security & Credentials (CRITICAL)

### 🚨 NEVER DO THIS:
- ❌ Do NOT save passwords in plain text files
- ❌ Do NOT hardcode credentials in Python scripts
- ❌ Do NOT share credentials in chat/email
- ❌ Do NOT store in config JSON with passwords

### ✅ CORRECT APPROACHES:

#### Option 1: Interactive Input (Recommended for non-technical users)
Scripts prompt each time — password is never saved. Simple and secure.

#### Option 2: Environment Variables (For frequent use)
```bash
# Terminal में एक बार दें (session के लिए)
export UDISE_USERNAME="your_username"
export UDISE_PASSWORD="your_password"
export UDISE_SCHOOL_CODE="24110206411"

# Script run करें
python3 ~/udise_student_manager.py

# Session बंद होने पर variables forget हो जाएंगे
```

**To make permanent** (still secure — hidden in home directory):
```bash
echo 'export UDISE_USERNAME="your_username"' >> ~/.bashrc
echo 'export UDISE_PASSWORD="your_password"' >> ~/.bashrc
source ~/.bashrc
```

#### Option 3: System Keyring (Most secure)
```bash
pip install keyring
python3 -c "import keyring; keyring.set_password('udise', 'username', 'password')"
```
Script retrieves from keyring automatically.

---

## 🎯 Language & Communication

**This skill serves non-technical users** (school clerks, administrators).

### Communication Guidelines:
- ✅ Use **simple Hindi/Gujarati** explanations first
- ✅ Avoid technical jargon (say "file" not "artifact", "open" not "invoke")
- ✅ Explain each step in plain language
- ✅ Provide both code and click alternatives when possible
- ❌ No advanced terminology unless user demonstrates understanding

**User profile example** (Piyush, Senior Clerk):
```yaml
role: Senior Clerk, Government Highschool
language: Gujarati, Hindi (limited English)
technical_level: Non-technical; uses Excel daily but not programming
needs: Ready-to-use tools, step-by-step guidance, minimal configuration
prefers: Practical examples over theory
```

---

## 🐛 Troubleshooting

### Chrome Driver Issues
**Problem:** "Chrome driver not found" or version mismatch

**Solution:**
```bash
pip uninstall webdriver-manager -y
pip install webdriver-manager
```

Or manually update:
```bash
python3 -c "from webdriver_manager.chrome import ChromeDriverManager; print(ChromeDriverManager().install())"
```

### Login Failures
**Problem:** Portal rejects credentials

**Checklist:**
- Username is usually school email or registered ID
- Password is case-sensitive
- School Code must be exactly 11 digits
- Internet connection active
- UDISE+ portal is online (https://udiseplus.gov.in/)

**If portal structure changed:** UDISE+ website may have updated. Inspect page elements (Right-click → Inspect) and share with developer to update selectors.

### Download Not Working
**Problem:** No data downloads, empty files

**Solution:**
1. Portal might require manual navigation — run with `headless=False` (default)
2. Check browser window during script run — which step fails?
3. Look for "Data not available" messages on portal
4. Some data requires specific date filters — set them manually first

### Data Format Issues
**Problem:** Columns are different, names mismatched

**Fix:** UDISE+ portal updates column names occasionally. Edit `udise_student_manager.py` → `scrape_table_data()` function to map column names correctly.

---

## 📈 Extended Features (Optional)

### Feature 1: Automated Monthly Scheduler

Set up cron job for automatic downloads:

```bash
crontab -e
```

Add line (runs on 1st of every month at 10 AM):
```
0 10 1 * * /home/piyush/UDISE_Data/venv/bin/python /home/piyush/udise_student_manager.py >> /home/piyush/UDISE_Data/cron.log 2>&1
```

**Test cron manually first:**
```bash
python3 ~/udise_student_manager.py --test-mode
```

*(Add `--test-mode` flag to script to skip actual download)*

---

### Feature 2: Report Cards as PDF

Install additional packages:
```bash
pip install weasyprint
```

Create `generate_pdf_reports.py`:
```python
import weasyprint
from jinja2 import Template

# Load HTML template, fill student data, convert to PDF
```

---

### Feature 3: Instagram-Ready Time Table Posts

Generate Instagram-ready images for sharing school schedules:

```python
from PIL import Image, ImageDraw, ImageFont

# Create 1080x1080 square image
width, height = 1080, 1080
img = Image.new('RGB', (width, height), color='#f0f4f8')
# ... add styled text, colors, school branding
img.save('instagram_time_table.jpg', quality=95)
```

**Use case:** Share school schedule on school's Instagram, WhatsApp groups, or notice boards.

**Customization options:**
- Add school logo
- Change colors (match school branding)
- Include contact details
- Add QR code linking to full schedule

---

### Feature 4: SMS Alerts for New Admissions

Use Twilio or Indian SMS gateway:
```python
from twilio.rest import Client

client = Client(account_sid, auth_token)
message = client.messages.create(
    body=f"New admission: {student_name}, Class {class}",
    from_='+1234567890',
    to='+919876543210'
)
```

---

### Feature 5: Dashboards with Charts

```python
import matplotlib.pyplot as plt

df = pd.read_excel('UDISE_Student_Master.xlsx')
df['Class'].value_counts().plot(kind='bar')
plt.savefig('class_distribution.png')
```

---

### Feature 6: School Time Table Generator

**Most requested by school clerks!** Creates practical time tables ready for Excel and Instagram.

#### Quick Start (Hindi/Gujarati users):
```bash
# Create time table Excel with standard school periods
python3 ~/generate_time_table.py

# Creates: ~/time_table.xlsx and ~/instagram_time_table.jpg
```

#### What You Get:
- **Excel file** (`time_table.xlsx`) — સમય સૂચિની સંપૂર્ણ ટેબલ, દરેક પાઠ/વિષય માટે
- **Instagram image** (`instagram_time_table.jpg`) — 1080×1080 પીક્સલ, સીધા મોકલી શકો છો

#### Sample Conversation Flow:
```
User: "Excel file instagram maa aap" - Wants to share Excel on Instagram
Agent: Time table Excel → PDF → Instagram image
Result: Practical workflow that works!
```

#### Customization (for your school):
1. **Periods timing** — Adjust in script:
   - ૮:૦૦-૮:૪૫, ૯:૫૦-૧૦:૩૫, etc.
2. **Subjects list** — Add/remove subjects in Excel
3. **Teacher names** — શિક્ષકોના નામ ઉમેરો

#### Instagram Sharing Steps:
1. Run the script → creates `instagram_time_table.jpg`
2. Open Instagram app
3. Tap "+" → select image
4. Add caption: "આજની સમય સૂચિ 📚 #SchoolSchedule"
5. Share to story/feed

**Pro tip:** Always save Excel template so you can regenerate image after edits!

---

### Feature 7: Bulk Student ID Generation

Auto-generate admission numbers for new students without IDs:

```python
import sys
sys.path.insert(0, '~/')
from excel_helper_functions import generate_student_id

df = pd.read_excel('UDISE_Student_Master.xlsx', sheet_name='Basic_Info')
new_ids = generate_student_id(df)
# new_ids = [(row_index, 'STU0001'), ...]
```

Assigns sequential IDs like `STU0001`, `STU0002`... avoiding duplicates.

---

## 🎯 Quick Reference Card

| Task | Command |
|------|---------|
| Activate environment | `cd ~/UDISE_Data && source venv/bin/activate` |
| Download data | `python3 ~/udise_student_manager.py` |
| Clean & analyze | `python3 ~/udise_data_processor.py` |
| Open master file | `libreoffice --calc ~/UDISE_Data/UDISE_Student_Master.xlsx` |
| Create backup | `cp ~/UDISE_Data/UDISE_Student_Master.xlsx ~/Backup_$(date +%Y%m%d).xlsx` |
| View log | `tail -f ~/UDISE_Data/cron.log` |

---

## 📚 References

### UDISE+ Portal Information
- **URL:** https://udiseplus.gov.in/
- **Purpose:** Unified District Information System for Education
- **Managed by:** Department of School Education, Government of India
- **Data includes:** Student enrollment, infrastructure, teacher details, financial data

### Session-Specific Resources

### Instagram Time Table Post (2026-05-08):
- `references/instagram-time-table-post.md` — complete caption, hashtags, upload steps
- `instagram_time_table.jpg` — ready-to-post 1080×1080 image

### Session Troubleshooting Guide (2026-05-08):
- `references/troubleshooting-session-2026-05-08.md` — handling file delivery issues, platform limitations, communication lessons
- `references/instagram-time-table-post.md` — complete caption, hashtags, upload steps
- `instagram_time_table.jpg` — ready-to-post 1080×1080 image

**Time Table Generator Corrected Script:**
- `references/time-table-session-notes.md` — fixed Python code, Excel formatting guide, troubleshooting notes

**Quick Scripts Reference:**
- `references/scripts-reference.md` — what each script does, when to run it
- `references/udise-portal-specifics.md` — portal structure, login page selectors if they change

**Excel Structure Template:**
- `templates/excel-structure.md` — master Excel schema reference card

---

### API Alternative (if available)
UDISE+ may offer REST API endpoints for registered schools. Check portal → "API Access" or contact admin.

If API exists, replace Selenium with:
```python
import requests
response = requests.get(api_endpoint, auth=(username, password), params={'school': school_code})
data = response.json()
```

---

## 🧠 Memory Triggers

**Save to memory after using this skill:**
- Portal changes (new login page structure)
- Column name changes in downloaded data
- New required fields in UDISE+ forms
- Browser/Chrome version compatibility issues
- User's specific school needs (Gujarati medium? Specific report formats?)

---

## 🆘 Getting Help

**If automation breaks:**
1. Check UDISE+ portal manually — is it accessible?
2. Inspect browser window during script run — which step fails?
3. Share error message + screenshot (if possible)
4. Update Selenium selectors in script (XPath, button IDs)

**Contact:** Document issues in `~/UDISE_Data/issues_log.md` for tracking.

---

## 📝 Notes for Non-Technical Users

**You don't need to understand code!**

Just remember:
- **Double-click** the Python script file to run (if configured) OR use terminal
- **Type your password** when asked (it won't show on screen)
- **Wait** for browser to finish (don't close it)
- **Open Excel** file to see your data
- **Save** your changes

**For help:** Call your tech support person with error message.

---

## 🎁 Session Artifacts (For Piyush - 2026-05-08)

The following files were created in your home directory during our conversation:

### Skills & Documentation
- `~/.hermes/skills/productivity/school-records-automation/SKILL.md` - Complete guide
- `~/UDISE_User_Guide.md` - Hindi/Gujarati user manual

### Python Scripts (in `~/`)
- `udise_student_manager.py` - Main download tool
- `udise_data_processor.py` - Data cleaner & analyzer
- `excel_helper_functions.py` - Excel utility functions

### Directories Created
- `~/UDISE_Data/` - All data will be saved here
  - Contains virtual environment `venv/`
  - Scripts should be copied here for use

### Quick Commands for You

```bash
# Run downloader
python3 ~/udise_student_manager.py

# Process data after download
python3 ~/udise_data_processor.py

# Open master Excel
libreoffice --calc ~/UDISE_Data/UDISE_Student_Master.xlsx
```

**Next step:** Ask your IT person to help install the Python packages (see "Setup" section above), or run the install commands yourself if comfortable.

---

*Last updated: 2026-05-08 | Created for Piyush Sahib, Government Highschool*