diff --git a/.claude/JOURNAL.md b/.claude/JOURNAL.md index 4f3c90f..1dd9976 100644 --- a/.claude/JOURNAL.md +++ b/.claude/JOURNAL.md @@ -237,3 +237,6 @@ This journal tracks substantive work on documents, diagrams, and documentation c 78. **Task - Activity sampler as independent JupyterHub service**: Converted from lazy-start to boot-time service
**Result**: **Problem**: Activity sampler only started when someone viewed Activity page - not truly independent. **Solution**: Created standalone `activity_sampler.py` service that runs as JupyterHub managed service (like idle-culler). Uses JupyterHub REST API with aiohttp to fetch user activity data. Configured with `activity-sampler-role` having `list:users`, `read:users:activity`, `read:servers` scopes. Service starts automatically on JupyterHub boot, runs continuously with 5-second initial delay. Records samples to same SQLite database (`/data/activity_samples.sqlite`). Removed lazy-start code from ActivityDataHandler. Added aiohttp to Dockerfile dependencies. Renamed env var to `JUPYTERHUB_ACTIVITYMON_SAMPLE_INTERVAL` for consistency + +79. **Task - Activity tracking methodology research**: Documented industry approaches for activity scoring
+ **Result**: Created `docs/activity-tracking-methodology.md` covering: (1) Exponential Moving Average with decay - our current approach using half-life parameterization, (2) Time-Window Percentage (Hubstaff) - active seconds per 10-min window with 60-80% typical for development, (3) Daily Target (8h=100%) - maps to work expectations, (4) GitHub Contribution Graph - threshold-based intensity levels. Research confirms our EMA approach is industry-standard. Key insight from Hubstaff: 100% activity is unrealistic, typical ranges 30-80% depending on role diff --git a/docs/activity-tracking-methodology.md b/docs/activity-tracking-methodology.md new file mode 100644 index 0000000..a02c447 --- /dev/null +++ b/docs/activity-tracking-methodology.md @@ -0,0 +1,226 @@ +# Activity Tracking Methodology Research + +## Current Implementation + +Our current approach uses **exponential decay scoring**: +- Samples collected every 10 minutes (configurable) +- Each sample marked active/inactive based on `last_activity` within threshold +- Score calculated as weighted ratio: `weighted_active / weighted_total` +- Weight formula: `weight = exp(-λ × age_hours)` where `λ = ln(2) / half_life` +- Default half-life: 24 hours (activity from yesterday worth 50%) + +## Industry Approaches + +### 1. Exponential Moving Average (EMA) / Time-Decay Systems + +**How it works:** +- Recent events weighted more heavily than older ones +- Decay factor (α) determines how quickly old data loses relevance +- Example: α=0.5 per day means yesterday's activity worth 50%, two days ago worth 25% + +**Half-life parameterization:** +- More intuitive than raw decay factor +- "Activity has a 24-hour half-life" is clearer than "α=0.5" +- Our implementation already uses this approach + +**Pros:** +- Memory-efficient (no need to store all historical data) +- Naturally handles irregular sampling intervals +- Smooths out noise/outliers + +**Cons:** +- Older activity never fully disappears (asymptotic to zero) +- May not match user intuition of "weekly activity" + +**Reference:** [Exponential Moving Averages at Scale](https://odsc.com/blog/exponential-moving-averages-at-scale-building-smart-time-decay-systems/) + +--- + +### 2. Time-Window Activity Percentage (Hubstaff approach) + +**How it works:** +- Fixed time window (e.g., 10 minutes) +- Count active seconds / total seconds = activity % +- Aggregate over day/week as average of windows + +**Hubstaff's formula:** +``` +Active seconds / 600 = activity rate % (per 10-min segment) +``` + +**Key insight from Hubstaff:** +> "Depending on someone's job and daily tasks, activity rates will vary widely. People with 75% scores and those with 25% scores can often times both be working productively." + +**Typical benchmarks:** +- Data entry/development: 60-80% keyboard/mouse activity +- Research/meetings: 30-50% activity +- 100% is unrealistic for any role + +**Pros:** +- Simple to understand +- Direct mapping to "how active was I today" + +**Cons:** +- Doesn't capture quality of work +- Penalizes reading, thinking, meetings + +**Reference:** [Hubstaff Activity Calculation](https://support.hubstaff.com/how-are-activity-levels-calculated/) + +--- + +### 3. Productivity Categorization (RescueTime approach) + +**How it works:** +- Applications/websites pre-categorized by productivity score (-2 to +2) +- Time spent in each category weighted and summed +- Daily productivity score = weighted sum / total time + +**Categories:** +- Very Productive (+2): IDE, documentation +- Productive (+1): Email, spreadsheets +- Neutral (0): Uncategorized +- Distracting (-1): News sites +- Very Distracting (-2): Social media, games + +**Pros:** +- Captures quality of activity, not just presence +- Customizable per user/role + +**Cons:** +- Requires app categorization (complex to implement) +- Subjective classification +- Not applicable to JupyterLab (all activity is "productive") + +**Reference:** [RescueTime Methodology](https://www.rescuetime.com/) + +--- + +### 4. GitHub Contribution Graph (Threshold-based intensity) + +**How it works:** +- Count contributions per day (commits, PRs, issues) +- Map counts to 4-5 intensity levels +- Levels based on percentiles of user's own activity + +**Typical thresholds:** +```javascript +// Example from implementations +thresholds: [0, 10, 20, 30] // contributions per day +colors: ['#ebedf0', '#9be9a8', '#40c463', '#30a14e', '#216e39'] +``` + +**Key insight:** +- Relative to user's own history (not absolute) +- Someone with 5 commits/day max sees different scale than 50 commits/day + +**Pros:** +- Visual, intuitive +- Adapts to user's activity patterns + +**Cons:** +- Binary daily view (no intra-day granularity) +- Doesn't show decay/trend + +--- + +### 5. Daily Target Approach (8h = 100%) + +**How it works:** +- Define expected activity hours per day (e.g., 8h) +- Actual active hours / expected hours = daily score +- Cap at 100% or allow overtime bonus + +**Formula:** +``` +Daily score = min(1.0, active_hours / 8.0) × 100 +Weekly score = avg(daily_scores) +``` + +**Pros:** +- Maps directly to work expectations +- Easy to explain to users + +**Cons:** +- Assumes consistent work schedule +- Doesn't account for part-time, weekends +- JupyterHub users may have variable schedules + +--- + +## Recommendations for JupyterHub Activity Monitor + +### Option A: Keep Current (EMA with decay) + +Our current implementation is actually well-designed for the use case: + +| Aspect | Current Implementation | +|--------|------------------------| +| Sampling | Every 10 min (configurable) | +| Active threshold | 60 min since last_activity | +| Decay | 24-hour half-life | +| Score range | 0-100% | +| Visualization | 5-segment bar with color coding | + +**Suggested improvements:** +1. Add tooltip showing actual score percentage +2. Consider longer half-life (48-72h) for less frequent users +3. Document what the score represents + +### Option B: Hybrid Daily + Decay + +Combine daily activity percentage with decay: + +```python +# Daily activity: hours active today / 8 hours (capped at 100%) +daily_score = min(1.0, active_hours_today / 8.0) + +# Apply decay to historical daily scores +weekly_score = sum(daily_score[i] * exp(-λ * i) for i in range(7)) / 7 +``` + +**Benefits:** +- More intuitive "8h = full day" concept +- Still decays older activity + +### Option C: Simplified Presence-Based + +For JupyterLab, activity mostly means "server running + recent kernel activity": + +| Status | Points/day | +|--------|------------| +| Offline | 0 | +| Online, idle > 1h | 0.25 | +| Online, idle 15m-1h | 0.5 | +| Online, active < 15m | 1.0 | + +Weekly score = sum of daily points / 7 + +--- + +## Decision Points + +1. **What does "100% activity" mean for JupyterHub users?** + - Option: Active during all sampled periods in retention window + - Option: 8 hours of activity per day + - Option: Relative to user's own historical average + +2. **How fast should old activity decay?** + - Current: 24-hour half-life (aggressive decay) + - Alternative: 72-hour half-life (more stable) + - Alternative: 7-day half-life (weekly trend) + +3. **Should weekends count differently?** + - Current: All days weighted equally + - Alternative: Exclude weekends from expected activity + +--- + +## Sources + +- [Exponential Moving Averages at Scale (ODSC)](https://odsc.com/blog/exponential-moving-averages-at-scale-building-smart-time-decay-systems/) +- [Exponential Smoothing (Wikipedia)](https://en.wikipedia.org/wiki/Exponential_smoothing) +- [Hubstaff Activity Calculation](https://support.hubstaff.com/how-are-activity-levels-calculated/) +- [How Time is Calculated in Hubstaff](https://support.hubstaff.com/how-is-time-tracked-and-calculated-in-hubstaff/) +- [RescueTime](https://www.rescuetime.com/) +- [EWMA Formula (Corporate Finance Institute)](https://corporatefinanceinstitute.com/resources/career-map/sell-side/capital-markets/exponentially-weighted-moving-average-ewma/) +- [Developer Productivity Metrics (Axify)](https://axify.io/blog/developer-productivity-metrics)