"
```
**Assertions:**
- Returns 200 (even if no new emails to sync)
- Response indicates sync status: "completed", "no_integration", or "synced"
- If email integration is connected → shows count of synced messages
- If not connected → graceful message (not 500 crash)
- Check what email sync model stores:
```bash
cd singoa-api && python -c "
import django, os; os.environ.setdefault('DJANGO_SETTINGS_MODULE','singoa_project.settings'); django.setup()
from integrations.models import EmailMessage, EmailThread
from authentication.models import User
u = User.objects.get(email='artest.construction@singoa.com')
messages = EmailMessage.objects.filter(user=u).count()
threads = EmailThread.objects.filter(user=u).count()
print(f'Email messages: {messages}, Threads: {threads}')
"
```
---
### Test 7.2: Simulate Customer Reply — Inbound Email Webhook
Simulate a customer reply using the SendGrid inbound webhook:
```bash
curl -s -X POST http://localhost:8000/api/communications/email-inbound/ \
-H "Content-Type: application/json" \
-d '{
"from": "billing@slowpayer.com",
"to": "artest.construction@singoa.com",
"subject": "Re: Payment Reminder: Invoice INV-SLOW-006",
"text": "Hi John, I apologize for the delay. We are processing the payment now and it should be sent by end of this week. Thank you for your patience.",
"html": "Hi John, I apologize for the delay. We are processing the payment now and it should be sent by end of this week. Thank you for your patience.
"
}'
```
**Assertions:**
- Returns 200 with `success: true` and `processed: true`
- Reply is stored in the system
- Reply is linked to the correct customer (SlowPayer LLC)
- Reply is linked to the correct invoice thread (INV-SLOW-006)
- `last_customer_response` on the customer is updated to now
---
### Test 7.3: Verify Hub Message Created for Inbound
```bash
curl -s http://localhost:8000/api/communications/hub/messages/ \
-H "Authorization: Bearer "
```
**Assertions:**
- Returns 200 with message list
- Contains entry for the simulated reply with:
- `sender_name`: "billing@slowpayer.com" or "SlowPayer LLC"
- `sender_email`: "billing@slowpayer.com"
- `subject`: "Re: Payment Reminder: Invoice INV-SLOW-006"
- `preview`: first ~100 chars of the message
- `classification`: "external_ar" or "payment_response" (NOT "internal")
- `classification_confidence`: > 0.5
- `status`: "pending" or "new"
- `customer_name`: "SlowPayer LLC" (linked)
- Click for detail:
```bash
curl -s http://localhost:8000/api/communications/hub/messages// \
-H "Authorization: Bearer "
```
Shows full content, AI analysis, suggested action
---
### Test 7.4: Verify AI Classification of Reply
The hub message detail should include AI analysis:
**Assertions on detail response:**
- `ai_summary`: summarizes the reply (e.g., "Customer promises payment by end of week")
- `ai_suggested_action`: recommended next step (e.g., "Log payment promise, reschedule reminders")
- `request_type`: classified type (e.g., "payment_promise", "dispute", "information_request")
- `extracted_entities`: parsed details (e.g., payment date, reference numbers)
- If AI classification is missing or generic → enhance the classification prompt
---
### Test 7.5: Simulate Payment Confirmation Reply
```bash
curl -s -X POST http://localhost:8000/api/communications/email-inbound/ \
-H "Content-Type: application/json" \
-d '{
"from": "pay@recentpayer.com",
"to": "artest.construction@singoa.com",
"subject": "Re: Invoice INV-RECENT-005",
"text": "Payment has been sent via wire transfer. Reference number: WT-2026-03-001. Please confirm receipt.",
"html": ""
}'
```
**Assertions:**
- Reply is classified as "payment_confirmation" or "payment_notification"
- System detects payment intent
- Extracted entities include reference number "WT-2026-03-001"
- Creates notification or approval for user to confirm the payment
- If auto-approve is enabled → automatically logs a payment promise
- Future scheduled reminders for this invoice should be reconsidered
---
### Test 7.6: Simulate Dispute Reply
```bash
curl -s -X POST http://localhost:8000/api/communications/email-inbound/ \
-H "Content-Type: application/json" \
-d '{
"from": "disputes@disputeking.com",
"to": "artest.construction@singoa.com",
"subject": "Re: Invoice INV-DISPUTE-002",
"text": "We are disputing this invoice. The materials delivered were not as specified in the purchase order. We request a credit memo for $3,000 of the $9,000 total.",
"html": ""
}'
```
**Assertions:**
- Reply classified as "dispute" or "billing_dispute"
- Creates ActionQueueItem for human review (disputes can't be auto-resolved)
- Invoice lifecycle_status should be updated to "disputed" (or flagged)
- Future automated reminders for this invoice should be BLOCKED
- Notification sent to user about the dispute
- AI summary captures the dispute amount ($3,000 of $9,000)
---
### Test 7.7: Event Rescheduling After Customer Reply
When a customer replies, future scheduled reminders should be reconsidered:
```bash
cd singoa-api && python -c "
import django, os; os.environ.setdefault('DJANGO_SETTINGS_MODULE','singoa_project.settings'); django.setup()
from automations.models import PlannedAction
from invoices.models import Customer
from authentication.models import User
u = User.objects.get(email='artest.construction@singoa.com')
slow = Customer.objects.get(user=u, customer_name='SlowPayer LLC')
# Check if future tasks were rescheduled/cancelled after the reply in Test 7.2
actions = PlannedAction.objects.filter(
user=u,
invoice__customer=slow,
status__in=['planned','scheduled','pending']
)
print(f'Remaining planned actions for SlowPayer: {actions.count()}')
for a in actions:
print(f' ID:{a.id} Scheduled:{a.scheduled_for} Status:{a.status}')
# The customer replied with a promise to pay by end of week
# So future reminders should be rescheduled or put on hold
"
```
**Assertions:**
- After customer reply with payment promise → future reminders are paused/rescheduled
- The system doesn't send another reminder tomorrow when customer just replied today
- If no rescheduling happens → the executor's pre-check (Test 3.3) should catch it at execution time
---
### Test 7.8: Process Customer Response via API
```bash
curl -s -X POST http://localhost:8000/api/automations/ar/invoice//response/ \
-H "Content-Type: application/json" \
-H "Authorization: Bearer " \
-d '{"response_text": "Will pay by Friday, sorry for the delay."}'
```
**Assertions:**
- Returns 200 with `success: true`
- `analysis` object contains:
- Detected intent (payment promise)
- Suggested actions
- Confidence score
- Customer's response is logged in communication history
- Future reminders adjusted based on the response
---
### Test 7.9: Frontend — Hub/Communications Page (Chrome MCP)
```
1. Navigate to /hub (logged in as construction user)
2. Take snapshot
```
**Assertions:**
- Hub page loads with message list
- Incoming messages displayed with:
- Sender name/email
- Subject line
- Preview text
- Classification badge (e.g., "AR Related", "Payment Promise", "Dispute")
- Time received
- Status (New, Reviewed, Responded)
- Click a message → shows full content with:
- Complete message text
- AI analysis sidebar or section
- Suggested response
- Action buttons (Reply, Classify, Route, Dismiss)
- Reply functionality works (type response, click send)
- Reclassify works (change classification category)
- Take screenshot for evidence
---
### Test 7.10: Hub Statistics
```bash
curl -s http://localhost:8000/api/communications/hub/stats/ \
-H "Authorization: Bearer "
```
**Assertions:**
- Returns 200 with `success: true`
- Stats include: `total_messages`, `pending_messages`, `classified_messages`
- `by_classification` breaks down by type
- `response_rate` is calculated correctly
- Numbers match the actual messages we've processed
**Phase 7 is PASS when ALL 10 tests pass. Update state tracker.**
---
## PHASE 8: APPROVAL & REVIEW WORKFLOWS
### Test 8.1: List Action Queue — Pending Items
```bash
curl -s "http://localhost:8000/api/automations/ar/action-queue/?status=pending" \
-H "Authorization: Bearer "
```
**Assertions:**
- Returns 200 with `success: true`
- `items` list contains pending items from previous phases:
- Disputed invoice items (from Phase 3/7)
- Email-related warnings (from Phase 1/3)
- Escalation items (from segmentation)
- Each item has ALL expected fields:
- `id`, `action_type`, `action_type_display`
- `priority` (low/medium/high/critical), `priority_display`
- `status` = "pending"
- `title` — human-readable title
- `description` — detailed description
- `ai_recommendation` — AI suggested action
- `ai_confidence` — confidence score (0-1)
- `quick_actions` — list of quick resolution options
- `customer_name`, `invoice_number` (if applicable)
- `total_outstanding` — amount at stake
- `created_at`, `expires_at`
- Items are sorted by priority (critical first)
---
### Test 8.2: Get Action Queue Item Detail
```bash
curl -s http://localhost:8000/api/automations/ar/action-queue// \
-H "Authorization: Bearer "
```
**Assertions:**
- Returns 200 with full item details
- `context_data` provides additional context (related invoices, customer history)
- `ai_reasoning` explains why this item was created
- `quick_actions` provides actionable options:
```json
[
{"action": "update_email", "label": "Update Email Address"},
{"action": "schedule_call", "label": "Schedule Phone Call"},
{"action": "payment_plan", "label": "Offer Payment Plan"}
]
```
---
### Test 8.3: Resolve Action Queue Item
```bash
curl -s -X POST http://localhost:8000/api/automations/ar/action-queue//resolve/ \
-H "Content-Type: application/json" \
-H "Authorization: Bearer " \
-d '{
"resolution": "Contacted customer by phone. Payment confirmed for next week.",
"notes": "Spoke with Sarah in AP dept. Wire transfer expected by March 7."
}'
```
**Assertions:**
- Returns 200 with `success: true`
- Item status changed to "completed"
- `resolved_by` = current user
- `resolved_at` = current timestamp
- `resolution` matches what was sent
- `resolution_notes` matches what was sent
- Activity feed shows resolution entry
- Audit log shows resolution entry
- If this was blocking a PlannedAction → that action can now proceed
---
### Test 8.4: Dismiss Action Queue Item
```bash
curl -s -X POST http://localhost:8000/api/automations/ar/action-queue//dismiss/ \
-H "Content-Type: application/json" \
-H "Authorization: Bearer " \
-d '{"reason": "Customer has been contacted offline, no further action needed."}'
```
**Assertions:**
- Returns 200 with `success: true`
- Item status changed to "dismissed"
- Reason is logged
- Related planned actions remain as-is (not automatically cancelled)
---
### Test 8.5: Quick Action — Update Customer Email
If there's an item about invalid email:
```bash
curl -s -X POST http://localhost:8000/api/automations/ar/action-queue//quick-action/ \
-H "Content-Type: application/json" \
-H "Authorization: Bearer " \
-d '{
"action": "update_email",
"data": {"email": "newemail@customer.com"}
}'
```
**Assertions:**
- Returns 200 with `success: true`
- Customer's email is actually updated in the database
- Item is resolved automatically
- Previously blocked PlannedAction is now unblocked
- Activity feed shows the email update
---
### Test 8.6: Review Queue — Unified View
```bash
curl -s http://localhost:8000/api/automations/review-tasks/ \
-H "Authorization: Bearer "
```
**Assertions:**
- Returns combined view of ActionQueueItems + ApprovalRequests
- Each task has: type, source, priority, title, actions available
- Sorted by priority and urgency
---
### Test 8.7: Test Notification on Approval-Required Events
```bash
# Trigger a test approval notification
curl -s -X POST http://localhost:8000/api/notifications/test-approval/ \
-H "Authorization: Bearer "
```
**Assertions:**
- Returns 200 with `success: true`
- In-app notification created:
```bash
curl -s http://localhost:8000/api/notifications/?is_read=false \
-H "Authorization: Bearer "
```
Should contain the approval notification with action buttons
- If email notifications enabled → email would be sent
- Notification contains action tokens for approve/reject
---
### Test 8.8: Execute Notification Action via Token
Get the notification with actions, then use the action token:
```bash
# Get notification detail with action tokens
curl -s http://localhost:8000/api/notifications// \
-H "Authorization: Bearer "
```
Extract action token, then:
```bash
curl -s -X POST http://localhost:8000/api/notifications/action// \
-H "Content-Type: application/json"
```
**Assertions:**
- Returns 200 with `success: true`
- Action is executed (approve/reject based on token type)
- Token is marked as used (can't be reused)
- Related items are updated accordingly
- Audit trail updated
---
### Test 8.9: Notification Preferences — Quiet Hours
```bash
# Set quiet hours
curl -s -X PUT http://localhost:8000/api/notifications/preferences/ \
-H "Content-Type: application/json" \
-H "Authorization: Bearer " \
-d '{
"quiet_hours_enabled": true,
"quiet_hours_start": "22:00",
"quiet_hours_end": "07:00",
"quiet_hours_timezone": "America/New_York"
}'
```
**Assertions:**
- Preferences saved correctly
- Verify by GET:
```bash
curl -s http://localhost:8000/api/notifications/preferences/ \
-H "Authorization: Bearer "
```
`quiet_hours_enabled`: true, start/end times match
- Notifications during quiet hours should be queued (not sent immediately)
- Notifications outside quiet hours sent normally
---
### Test 8.10: Frontend — Review Queue Page (Chrome MCP)
```
1. Navigate to /automations/review-queue
2. Take snapshot
```
**Assertions:**
- Page loads with list of pending review items
- Each item shows: type icon, title, priority badge, customer, description
- Approve button visible and functional
- Reject/Dismiss button visible and functional
- Bulk selection checkbox exists
- Bulk approve button works
- Filter by status works (pending, completed, dismissed)
- Filter by type works
- Filter by confidence works
- Search functionality exists
- Item count matches API response
- Take screenshot for evidence
---
### Test 8.11: Bulk Approve Multiple Items
```bash
curl -s -X POST http://localhost:8000/api/automations/review-queue/bulk-approve/ \
-H "Content-Type: application/json" \
-H "Authorization: Bearer " \
-d '{"item_ids": ["", ""]}'
```
**Assertions:**
- Returns 200 with `approved_count: 2`
- All specified items are now approved/completed
- Each approval logged in audit trail
- Activity feed updated for each
---
### Test 8.12: Unread Notification Count
```bash
curl -s http://localhost:8000/api/notifications/unread_count/ \
-H "Authorization: Bearer "
```
**Assertions:**
- Returns `count` matching actual unread notifications
- After marking all read:
```bash
curl -s -X POST http://localhost:8000/api/notifications/mark-all-read/ \
-H "Content-Type: application/json" \
-H "Authorization: Bearer "
```
Count drops to 0
**Phase 8 is PASS when ALL 12 tests pass. Update state tracker.**
---
## PHASE 9: SETTINGS & CONFIGURATION
### Test 9.1: Get Business Rules
```bash
curl -s http://localhost:8000/api/automations/ar/business-rules/ \
-H "Authorization: Bearer "
```
**Assertions:**
- Returns 200 with full BusinessRuleSet
- ALL fields present:
- `due_soon_days`: 7
- `overdue_2_start_days`: 8
- `overdue_3_start_days`: 15
- `critical_threshold_days`: 30
- `first_reminder_days`: 1
- `reminder_frequency_days`: 7
- `tone_overdue_1`: "friendly", `tone_overdue_1_display`: human-readable
- `tone_overdue_2`: "professional"
- `tone_overdue_3`: "firm"
- `tone_critical`: "urgent"
- `send_emails_start_hour`: 9
- `send_emails_end_hour`: 17
- `working_days`: [0,1,2,3,4]
- `promise_grace_days`: 2
- `late_fee_enabled`, `late_fee_percent`, `late_fee_grace_days`
- `ar_workflow_hour`, `reminder_window_1_hour`, `reminder_window_2_hour`
---
### Test 9.2: Update Business Rules
```bash
curl -s -X PUT http://localhost:8000/api/automations/ar/business-rules/ \
-H "Content-Type: application/json" \
-H "Authorization: Bearer " \
-d '{
"reminder_frequency_days": 5,
"critical_threshold_days": 45,
"send_emails_start_hour": 8,
"send_emails_end_hour": 18
}'
```
**Assertions:**
- Returns 200
- Updated fields reflect new values
- Non-updated fields retain original values
- Verify: `GET /api/automations/ar/business-rules/` shows new values
- Next planner run uses updated rules (re-run planner and check)
**Revert changes after test:**
```bash
curl -s -X PUT http://localhost:8000/api/automations/ar/business-rules/ \
-H "Content-Type: application/json" \
-H "Authorization: Bearer " \
-d '{"reminder_frequency_days": 7, "critical_threshold_days": 30, "send_emails_start_hour": 9, "send_emails_end_hour": 17}'
```
---
### Test 9.3: Business Rules Validation — Reject Invalid Values
```bash
# Negative days
curl -s -X PUT http://localhost:8000/api/automations/ar/business-rules/ \
-H "Content-Type: application/json" \
-H "Authorization: Bearer " \
-d '{"first_reminder_days": -1}'
```
**Assertions:** Returns 400 with validation error
```bash
# overdue_2 < overdue_1 threshold (illogical)
curl -s -X PUT http://localhost:8000/api/automations/ar/business-rules/ \
-H "Content-Type: application/json" \
-H "Authorization: Bearer " \
-d '{"overdue_2_start_days": 3, "overdue_3_start_days": 2}'
```
**Assertions:** Returns 400 (stage 3 can't start before stage 2)
```bash
# start_hour > end_hour
curl -s -X PUT http://localhost:8000/api/automations/ar/business-rules/ \
-H "Content-Type: application/json" \
-H "Authorization: Bearer " \
-d '{"send_emails_start_hour": 20, "send_emails_end_hour": 8}'
```
**Assertions:** Returns 400 or handles gracefully (overnight window)
---
### Test 9.4: Industry Default Business Rules
```bash
curl -s "http://localhost:8000/api/automations/ar/business-rules/industry-defaults/?industry=construction" \
-H "Authorization: Bearer "
```
**Assertions:**
- Returns 200 with industry-specific defaults
- Construction defaults differ from generic (e.g., longer payment cycles)
- Test for each industry:
```bash
for industry in construction education wholesale general; do
echo "=== $industry ==="
curl -s "http://localhost:8000/api/automations/ar/business-rules/industry-defaults/?industry=$industry" \
-H "Authorization: Bearer "
echo ""
done
```
- Each industry has reasonable, distinct defaults
- If all industries return identical defaults → enhance with industry-specific values
---
### Test 9.5: Get & Update Communication Settings
```bash
curl -s http://localhost:8000/api/auth/settings/communication/ \
-H "Authorization: Bearer "
```
**Assertions:**
- Returns 200 with all communication settings
- `sender_name`, `email_signature`, `business_tone`, `custom_instructions` all present and matching what we set in Phase 0
**Update and verify:**
```bash
curl -s -X PUT http://localhost:8000/api/auth/settings/communication/ \
-H "Content-Type: application/json" \
-H "Authorization: Bearer " \
-d '{
"sender_name": "John Builder - AR Dept",
"custom_instructions": "Always mention our 2% early payment discount for invoices paid within 10 days."
}'
```
**Assertions:**
- Returns 200 with updated values
- `sender_name` now shows "John Builder - AR Dept"
- `custom_instructions` updated
- Next generated email reflects these changes (run a quick reminder and check)
---
### Test 9.6: Get & Update Notification Preferences
```bash
curl -s http://localhost:8000/api/notifications/preferences/ \
-H "Authorization: Bearer "
```
**Assertions:**
- Returns 200 with all preference fields
- `enable_websocket`, `enable_email`, `enable_sms`, `enable_push` all present
- `quiet_hours_enabled`, `quiet_hours_start`, `quiet_hours_end` present
**Update:**
```bash
curl -s -X PUT http://localhost:8000/api/notifications/preferences/ \
-H "Content-Type: application/json" \
-H "Authorization: Bearer " \
-d '{
"enable_email": true,
"enable_sms": false,
"email_digest_enabled": true,
"email_digest_frequency": "daily",
"digest_send_hour": 8
}'
```
**Assertions:**
- Returns 200 with updated preferences
- Changes are saved and returned correctly
---
### Test 9.7: Collection Stages Configuration
```bash
curl -s http://localhost:8000/api/automations/ar/collection-stages/ \
-H "Authorization: Bearer "
```
**Assertions:**
- Returns 200 with list of collection stages
- Stages represent the escalation path (e.g., Friendly → Professional → Firm → Urgent → Collections)
- Each stage has: name, days threshold, tone, actions
---
### Test 9.8: Frontend — AI Automation Settings Page (Chrome MCP)
```
1. Navigate to /settings/ai-automation
2. Take snapshot
```
**Assertions:**
- Page loads with current automation rules
- Natural language chat interface visible (for creating rules)
- Can type a rule like "Send a reminder 3 days after an invoice is overdue"
- AI parses the rule
- Confirmation shown
- Rule is saved
- Existing rules listed with activate/deactivate toggles
- Templates section shows pre-built rule templates
- Can create rule from template
- All toggles and buttons functional
- No console errors
- Take screenshot
---
### Test 9.9: Frontend — Notification Settings Page (Chrome MCP)
```
1. Navigate to /settings/notifications
2. Take snapshot
```
**Assertions:**
- Channel configuration visible (Email, SMS, Push, WebSocket toggles)
- Toggle switches work (click to enable/disable)
- Quiet hours section with start/end time pickers
- Digest settings (frequency dropdown, send hour)
- Changes save when clicking save button
- No console errors
- Take screenshot
---
### Test 9.10: Natural Language Rule Creation
```bash
curl -s -X POST http://localhost:8000/api/automations/rules/parse/ \
-H "Content-Type: application/json" \
-H "Authorization: Bearer " \
-d '{"rule_text": "Send a friendly reminder 3 days after an invoice becomes overdue"}'
```
**Assertions:**
- Returns 200 with `success: true`
- `parsed_rule` contains structured rule data:
- Trigger: "invoice_overdue"
- Delay: 3 days
- Action: "send_reminder"
- Tone: "friendly"
- Rule can be saved and activated
**Phase 9 is PASS when ALL 10 tests pass. Update state tracker.**
---
## PHASE 10: DASHBOARD & FRONTEND INTEGRATION
### Test 10.1: Dashboard KPIs Accuracy (Chrome MCP)
```
1. Navigate to /dashboard
2. Take snapshot
3. Note all KPI values displayed
```
**Verify KPIs against actual data:**
```bash
cd singoa-api && python -c "
import django, os; os.environ.setdefault('DJANGO_SETTINGS_MODULE','singoa_project.settings'); django.setup()
from invoices.models import Invoice
from authentication.models import User
from django.db.models import Sum, Count
from decimal import Decimal
u = User.objects.get(email='artest.construction@singoa.com')
invoices = Invoice.objects.filter(user=u)
total_outstanding = invoices.exclude(status='paid').aggregate(total=Sum('amount'))['total'] or 0
overdue_amount = invoices.filter(status='overdue').aggregate(total=Sum('amount'))['total'] or 0
overdue_count = invoices.filter(status='overdue').count()
paid_count = invoices.filter(status='paid').count()
total_count = invoices.count()
print(f'Total Outstanding: \${total_outstanding}')
print(f'Overdue Amount: \${overdue_amount}')
print(f'Overdue Count: {overdue_count}')
print(f'Paid Count: {paid_count}')
print(f'Total Invoices: {total_count}')
print(f'Collection Rate: {paid_count/total_count*100:.1f}%' if total_count else 'N/A')
"
```
**Assertions:**
- Dashboard "Total Outstanding" matches calculated total_outstanding
- Dashboard "Overdue Amount" matches calculated overdue_amount
- Dashboard "Overdue Invoices" count matches overdue_count
- DSO calculation is present and reasonable
- Aging buckets (0-30, 31-60, 61-90, 90+) add up to total overdue
- At-risk customers list matches RISK/DELINQUENT segments
- If any KPI is wrong → verify the dashboard API endpoint and fix
---
### Test 10.2: Dashboard API Endpoint
```bash
curl -s http://localhost:8000/api/dashboard/ \
-H "Authorization: Bearer "
```
**Assertions:**
- Returns 200 with dashboard data
- Contains: outstanding amounts, overdue counts, aging breakdown, recent activity
- Numbers match the manual calculation above
---
### Test 10.3: Activity Feed on Dashboard (Chrome MCP)
**Assertions:**
- Activity feed widget visible on dashboard
- Shows recent events: emails sent, replies received, actions resolved
- Each entry has timestamp (relative: "5 minutes ago")
- Clickable entries navigate to relevant detail page
- "View all" link works
---
### Test 10.4: Scheduled Actions Widget on Dashboard
**Assertions:**
- Upcoming scheduled actions shown (next 5-10)
- Each shows: action type, customer name, scheduled time
- Completed actions NOT shown in upcoming
- Overdue/past-due actions highlighted
---
### Test 10.5: AI Insights Widget
```bash
curl -s http://localhost:8000/api/automations/agent/insights/ \
-H "Authorization: Bearer "
```
**Assertions:**
- Returns insights list (may be empty if not generated yet)
- Trigger generation if needed:
```bash
cd singoa-api && python -c "
import django, os; os.environ.setdefault('DJANGO_SETTINGS_MODULE','singoa_project.settings'); django.setup()
from automations.tasks.celery_tasks import generate_daily_ai_insights
result = generate_daily_ai_insights()
print(f'Result: {result}')
"
```
- Insights are contextual (reference actual customer data, not generic advice)
- Recommendations are actionable
- On dashboard: insights widget shows top 3-5 insights
---
### Test 10.6: Customer Detail Page — Full View (Chrome MCP)
```
1. Navigate to /customers/
2. Take snapshot
```
**Assertions:**
- Customer info displayed: name, email, phone, segment badge
- Outstanding amount shown
- Invoice list with statuses
- Communication timeline with:
- Sent reminders (with dates and content preview)
- Customer replies (with classification)
- Status changes
- Payment history
- AI insights for this customer
- No broken layouts or missing data
- Take screenshot
---
### Test 10.7: Invoice Detail Page (Chrome MCP)
```
1. Navigate to /invoices (list)
2. Click on a specific overdue invoice
3. Take snapshot
```
**Assertions:**
- Invoice details: number, amount, due date, status badge, customer
- Reminder history section showing:
- Each reminder sent (date, tone, subject)
- Customer responses (if any)
- Next scheduled reminder
- Payment history (if any partial payments)
- Action buttons: Send Reminder, Pause Automation, Mark as Paid
- Take screenshot
---
### Test 10.8: Automations Page — Complete View (Chrome MCP)
```
1. Navigate to /automations or relevant automations tab
2. Take snapshot of each section/tab
```
**Assertions:**
- Timeline/Scheduled tab: shows all planned actions with dates
- Action Queue tab: shows items needing review
- Activity Feed tab: shows recent automation events
- Rules/Settings tab: shows configured automation rules
- All tabs load without errors
- Data matches API responses
- Navigation between tabs works smoothly
- Take screenshots of each tab
---
### Test 10.9: Frontend Console Error Check (Chrome MCP)
```
1. Navigate through ALL key pages: /dashboard, /invoices, /customers, /automations/review-queue, /settings/ai-automation, /settings/notifications, /hub
2. For each page, check console for errors
```
**Assertions:**
- ZERO JavaScript errors on any page
- ZERO failed API calls (network tab shows all 200s)
- No "undefined" or "null" displayed where data should be
- No broken images or missing assets
- All pages render within 3 seconds
- If errors found → document and fix
**Phase 10 is PASS when ALL 9 tests pass. Update state tracker.**
---
## PHASE 11: CONTINUOUS CYCLE & EDGE CASES
### Test 11.1: Full Cycle End-to-End (Single Invoice Journey)
Execute the COMPLETE lifecycle for ONE invoice through the entire system:
**Step 1: Create a new overdue invoice**
```bash
curl -s -X POST http://localhost:8000/api/invoices/ \
-H "Content-Type: application/json" \
-H "Authorization: Bearer " \
-d '{
"customer": ,
"invoice_number": "INV-E2E-CYCLE-001",
"amount": 7500.00,
"due_date": "<5_days_ago>",
"status": "overdue",
"description": "Full E2E cycle test invoice"
}'
```
**Assert:** Invoice created (201), appears in list.
**Step 2: Run AR workflow**
```bash
curl -s -X POST http://localhost:8000/api/automations/ar/workflow/run/ \
-H "Content-Type: application/json" \
-H "Authorization: Bearer "
```
**Assert:** PlannedAction created for this invoice with tone "friendly" (5 days overdue).
**Step 3: Execute due tasks**
```bash
curl -s -X POST http://localhost:8000/api/automations/scheduled/execute/ \
-H "Content-Type: application/json" \
-H "Authorization: Bearer "
```
**Assert:** Task executed (or attempted), status changes to completed.
**Step 4: Verify email was generated by Collection Psychologist**
```bash
cd singoa-api && python -c "
import django, os; os.environ.setdefault('DJANGO_SETTINGS_MODULE','singoa_project.settings'); django.setup()
from automations.models import AutomationLog
log = AutomationLog.objects.filter(invoice__invoice_number='INV-E2E-CYCLE-001').last()
if log:
print(f'Status: {log.status}')
print(f'Subject: {log.subject}')
print(f'Body preview: {log.body[:200]}')
"
```
**Assert:** Email generated with personalized content, correct tone.
**Step 5: Verify communication logged everywhere**
Check: AutomationLog, ActivityFeed, AuditLog, customer history.
**Assert:** All 4 logging systems have entries.
**Step 6: Simulate customer reply**
```bash
curl -s -X POST http://localhost:8000/api/communications/email-inbound/ \
-H "Content-Type: application/json" \
-d '{
"from": "billing@slowpayer.com",
"to": "artest.construction@singoa.com",
"subject": "Re: INV-E2E-CYCLE-001",
"text": "We will process payment by end of this week.",
"html": ""
}'
```
**Assert:** Reply detected, classified, logged in hub.
**Step 7: Verify schedule updated**
**Assert:** Future reminders for this invoice are reconsidered.
**Step 8: Simulate payment received**
```bash
cd singoa-api && python -c "
import django, os; os.environ.setdefault('DJANGO_SETTINGS_MODULE','singoa_project.settings'); django.setup()
from invoices.models import Invoice
from django.utils import timezone
inv = Invoice.objects.get(invoice_number='INV-E2E-CYCLE-001')
inv.status = 'paid'
inv.paid_amount = inv.amount
inv.paid_date = timezone.now().date()
inv.save()
print(f'Invoice marked as paid')
"
```
**Assert:** All remaining planned actions cancelled. Dashboard metrics updated.
**Step 9: Verify dashboard reflects everything**
Check dashboard KPIs — outstanding reduced, paid count increased.
**Step 10: Full cycle complete**
**Assert:** Invoice went from creation → planning → execution → email → reply → payment → closed, with full audit trail at every step.
---
### Test 11.2: Invoice Status Transition Full Escalation
Create an invoice and simulate it aging through ALL escalation stages:
```bash
cd singoa-api && python -c "
import django, os; os.environ.setdefault('DJANGO_SETTINGS_MODULE','singoa_project.settings'); django.setup()
from invoices.models import Invoice, Customer
from automations.models import PlannedAction, AutomationLog
from authentication.models import User
from django.utils import timezone
from datetime import timedelta
u = User.objects.get(email='artest.construction@singoa.com')
c = Customer.objects.get(user=u, customer_name='SlowPayer LLC')
# Create test invoice
inv, _ = Invoice.objects.get_or_create(
user=u, customer=c, invoice_number='INV-ESCALATION-001',
defaults={'amount': 5000, 'due_date': timezone.now().date() - timedelta(days=1), 'status': 'overdue'}
)
stages = [
(1, 'friendly', 'Due Soon / Just Overdue'),
(8, 'professional', 'Overdue Stage 2'),
(15, 'firm', 'Overdue Stage 3'),
(30, 'urgent', 'Critical'),
(90, 'urgent', 'Final Notice / Collections'),
]
for days, expected_tone, stage_name in stages:
inv.due_date = (timezone.now() - timedelta(days=days)).date()
inv.save()
# Run planner
from automations.services.strategy_planner import ARStrategyPlanner
planner = ARStrategyPlanner(u)
result = planner.analyze_and_plan()
# Check what was planned
pa = PlannedAction.objects.filter(invoice=inv).order_by('-created_at').first()
actual_tone = getattr(pa, 'tone', 'N/A') if pa else 'NO ACTION'
print(f'{stage_name} ({days}d overdue): Expected={expected_tone}, Got={actual_tone}, Match={actual_tone==expected_tone}')
"
```
**Assertions:**
- Each stage produces the correct tone
- Escalation is progressive (never goes from urgent back to friendly)
- At each stage, email content is appropriate for the urgency level
---
### Test 11.3: Payment Received Mid-Cycle — Cancels Future Reminders
**Setup:** Invoice with scheduled reminders → receives payment:
```bash
cd singoa-api && python -c "
import django, os; os.environ.setdefault('DJANGO_SETTINGS_MODULE','singoa_project.settings'); django.setup()
from invoices.models import Invoice
from automations.models import PlannedAction
from authentication.models import User
from django.utils import timezone
u = User.objects.get(email='artest.construction@singoa.com')
inv = Invoice.objects.filter(user=u, status='overdue').first()
# Count planned actions before
before = PlannedAction.objects.filter(invoice=inv, status__in=['planned','scheduled','pending']).count()
# Pay the invoice
inv.status = 'paid'
inv.paid_amount = inv.amount
inv.paid_date = timezone.now().date()
inv.save()
# Run executor (should detect payment and cancel)
from automations.tasks.celery_tasks import execute_due_planned_actions
execute_due_planned_actions()
# Count after
after = PlannedAction.objects.filter(invoice=inv, status__in=['planned','scheduled','pending']).count()
cancelled = PlannedAction.objects.filter(invoice=inv, status='cancelled').count()
print(f'Before payment: {before} planned actions')
print(f'After payment: {after} planned, {cancelled} cancelled')
# Revert for other tests
inv.status = 'overdue'
inv.paid_amount = 0
inv.save()
"
```
**Assertions:**
- All planned actions for paid invoice are cancelled (after=0)
- Cancelled count matches before count
- No email sent after payment
---
### Test 11.4: Partial Payment Handling
```bash
cd singoa-api && python -c "
import django, os; os.environ.setdefault('DJANGO_SETTINGS_MODULE','singoa_project.settings'); django.setup()
from invoices.models import Invoice
from authentication.models import User
from django.utils import timezone
u = User.objects.get(email='artest.construction@singoa.com')
inv = Invoice.objects.filter(user=u, status='overdue', amount__gte=5000).first()
if inv:
# Partial payment: 60% of total
inv.paid_amount = inv.amount * 0.6
inv.status = 'partial'
inv.save()
print(f'Invoice {inv.invoice_number}: Amount={inv.amount}, Paid={inv.paid_amount}, Remaining={inv.amount - inv.paid_amount}')
# Run workflow
from automations.services.strategy_planner import ARStrategyPlanner
planner = ARStrategyPlanner(u)
result = planner.analyze_and_plan()
print(f'Plan result: {result}')
# Check — reminders should continue for remaining amount
from automations.models import PlannedAction
actions = PlannedAction.objects.filter(invoice=inv, status__in=['planned','scheduled','pending'])
print(f'Planned actions for partial invoice: {actions.count()}')
"
```
**Assertions:**
- Invoice status updated to "partial"
- Reminders continue for the REMAINING amount (not full amount)
- Generated email references the partial payment ("We received $3,000 of your $5,000 invoice...")
- Outstanding amount in email is correct ($2,000 remaining)
---
### Test 11.5: Multiple Invoices for Same Customer — Consolidation
```bash
cd singoa-api && python -c "
import django, os; os.environ.setdefault('DJANGO_SETTINGS_MODULE','singoa_project.settings'); django.setup()
from invoices.models import Invoice, Customer
from automations.models import PlannedAction
from authentication.models import User
from django.utils import timezone
from datetime import timedelta
u = User.objects.get(email='artest.construction@singoa.com')
c = Customer.objects.get(user=u, customer_name='Unreliable Inc')
# Ensure multiple overdue invoices exist
overdue = Invoice.objects.filter(customer=c, status='overdue').count()
print(f'Overdue invoices for Unreliable Inc: {overdue}')
# Run planner
from automations.services.strategy_planner import ARStrategyPlanner
planner = ARStrategyPlanner(u)
result = planner.analyze_and_plan()
# Check — should we get 1 consolidated reminder or multiple?
actions = PlannedAction.objects.filter(
user=u,
invoice__customer=c,
status__in=['planned','scheduled','pending']
)
print(f'Planned actions: {actions.count()}')
for a in actions:
print(f' Invoice: {a.invoice.invoice_number}, Scheduled: {a.scheduled_for}')
"
```
**Assertions:**
- Either: ONE consolidated email mentioning ALL overdue invoices (preferred)
- Or: Separate emails but properly spaced (not all at same time)
- Verify email content mentions all overdue amounts if consolidated
- Customer doesn't receive 5 separate emails in 5 minutes
---
### Test 11.6: Timezone Handling
```bash
cd singoa-api && python -c "
import django, os; os.environ.setdefault('DJANGO_SETTINGS_MODULE','singoa_project.settings'); django.setup()
from automations.models import PlannedAction, BusinessRuleSet
from authentication.models import User
from django.utils import timezone
import pytz
u = User.objects.get(email='artest.construction@singoa.com')
rules = BusinessRuleSet.objects.get(user=u)
# Check timezone handling
print(f'Server timezone: {timezone.get_current_timezone()}')
print(f'Business hours: {rules.send_emails_start_hour}:00 - {rules.send_emails_end_hour}:00')
actions = PlannedAction.objects.filter(user=u, status__in=['planned','scheduled','pending'])[:5]
for a in actions:
local_time = timezone.localtime(a.scheduled_for)
print(f'Action {a.id}: UTC={a.scheduled_for}, Local={local_time}, Hour={local_time.hour}')
if not (rules.send_emails_start_hour <= local_time.hour <= rules.send_emails_end_hour):
print(f' *** WARNING: Outside business hours in local time ***')
"
```
**Assertions:**
- All scheduled times are stored as UTC in database
- When converted to user's local timezone, they fall within business hours
- Frontend displays times in user's timezone (not UTC)
---
### Test 11.7: High Volume Stress Test
```bash
cd singoa-api && python -c "
import django, os; os.environ.setdefault('DJANGO_SETTINGS_MODULE','singoa_project.settings'); django.setup()
from invoices.models import Invoice, Customer
from authentication.models import User
from django.utils import timezone
from datetime import timedelta
import time
u = User.objects.get(email='artest.generic@singoa.com')
c = Customer.objects.filter(user=u).first()
# Create 50 overdue invoices
for i in range(50):
Invoice.objects.get_or_create(
user=u, customer=c, invoice_number=f'INV-STRESS-{i+1:03d}',
defaults={
'amount': 1000 + i * 50,
'due_date': (timezone.now() - timedelta(days=5+i%30)).date(),
'status': 'overdue',
}
)
total = Invoice.objects.filter(user=u, status='overdue').count()
print(f'Total overdue invoices: {total}')
# Time the workflow
start = time.time()
from automations.services.strategy_planner import ARStrategyPlanner
planner = ARStrategyPlanner(u)
result = planner.analyze_and_plan()
elapsed = time.time() - start
print(f'Planning took: {elapsed:.2f} seconds')
print(f'Result: {result}')
# Assert performance
if elapsed > 60:
print(f'*** FAILURE: Planning took {elapsed:.0f}s (limit: 60s) ***')
else:
print(f'OK: Planning completed in {elapsed:.1f}s')
"
```
**Assertions:**
- Planning 50+ invoices completes in < 60 seconds
- No timeout errors
- No memory issues
- All invoices planned correctly
- Clean up stress test data after
---
### Test 11.8: Error Recovery — Failed Email Retry
```bash
cd singoa-api && python -c "
import django, os; os.environ.setdefault('DJANGO_SETTINGS_MODULE','singoa_project.settings'); django.setup()
from automations.models import PlannedAction
from authentication.models import User
from django.utils import timezone
u = User.objects.get(email='artest.construction@singoa.com')
# Find a failed task (or create one)
pa = PlannedAction.objects.filter(user=u, status='failed').first()
if not pa:
pa = PlannedAction.objects.filter(user=u, status__in=['planned','scheduled']).first()
if pa:
pa.status = 'failed'
pa.save()
print(f'Marked task {pa.id} as failed for retry test')
if pa:
# Retry mechanism should pick up failed tasks
pa.status = 'planned' # Reset to allow retry
pa.execution_lock = None
pa.scheduled_for = timezone.now()
pa.save()
from automations.tasks.celery_tasks import execute_due_planned_actions
result = execute_due_planned_actions()
print(f'Retry result: {result}')
pa.refresh_from_db()
print(f'Task {pa.id} status after retry: {pa.status}')
"
```
**Assertions:**
- Failed tasks can be retried
- Retry doesn't create duplicate sends
- If task fails again, it's marked failed (not stuck in loop)
- Maximum retry count should be enforced (e.g., max 3 retries)
---
### Test 11.9: Data Integrity Under Concurrent Operations
```bash
cd singoa-api && python -c "
import django, os; os.environ.setdefault('DJANGO_SETTINGS_MODULE','singoa_project.settings'); django.setup()
from invoices.models import Invoice
from automations.models import PlannedAction, AutomationLog
from authentication.models import User
from django.utils import timezone
from django.db import IntegrityError
u = User.objects.get(email='artest.construction@singoa.com')
# Verify no orphaned records
orphaned_actions = PlannedAction.objects.filter(user=u, invoice__isnull=True).count()
orphaned_logs = AutomationLog.objects.filter(user=u, invoice__isnull=True).count()
print(f'Orphaned PlannedActions (no invoice): {orphaned_actions}')
print(f'Orphaned AutomationLogs (no invoice): {orphaned_logs}')
# Verify no duplicate PlannedActions
from django.db.models import Count
dupes = PlannedAction.objects.filter(user=u, status__in=['planned','scheduled','pending']).values('invoice_id','action_type').annotate(cnt=Count('id')).filter(cnt__gt=1)
print(f'Duplicate PlannedActions: {dupes.count()}')
for d in dupes:
print(f' Invoice {d[\"invoice_id\"]}: {d[\"cnt\"]} duplicates')
if orphaned_actions == 0 and orphaned_logs == 0 and dupes.count() == 0:
print('OK: Data integrity verified')
else:
print('*** FAILURES detected — investigate and fix ***')
"
```
**Assertions:**
- Zero orphaned records
- Zero duplicate planned actions
- All foreign keys valid
- No data corruption after all our testing
---
### Test 11.10: Daily Digest Generation
```bash
cd singoa-api && python -c "
import django, os; os.environ.setdefault('DJANGO_SETTINGS_MODULE','singoa_project.settings'); django.setup()
from automations.tasks.celery_tasks import generate_daily_ai_insights
result = generate_daily_ai_insights()
print(f'Insights result: {result}')
"
```
```bash
curl -s http://localhost:8000/api/notifications/digests/ \
-H "Authorization: Bearer "
```
**Assertions:**
- Digest is generated or can be generated
- Contains summary of: emails sent, payments matched, approvals created
- `is_empty` is false (we have activity)
- Digest detail shows actionable summary
**Phase 11 is PASS when ALL 10 tests pass. Update state tracker.**
---
## PHASE 12: FINAL VERIFICATION & PRODUCTION READINESS
### Test 12.1: Complete Workflow Walkthrough — Construction User
Walk through the ENTIRE workflow as the construction user. Using BOTH API calls and Chrome MCP frontend:
1. **Login** → `POST /api/auth/login/` → get token → Navigate to /dashboard in Chrome
2. **Verify dashboard** → KPIs correct, activity feed populated, scheduled actions visible
3. **Check overdue invoices** → Navigate to /invoices → filter by overdue → all present with correct statuses
4. **Check automation settings** → Navigate to /settings/ai-automation → rules configured
5. **Run AR workflow** → `POST /api/automations/ar/workflow/run/` → plans created
6. **View planned actions** → Navigate to automations page → timeline shows correct schedule
7. **Execute tasks** → `POST /api/automations/scheduled/execute/` → emails sent
8. **Check email quality** → Read the generated emails → verify construction-specific language, personalization
9. **View communication log** → Navigate to customer detail → communication timeline shows sent emails
10. **Simulate reply** → `POST /api/communications/email-inbound/` → reply detected and classified
11. **Check approval queue** → Navigate to /automations/review-queue → items requiring attention
12. **Resolve items** → Click approve/resolve → items cleared
13. **Verify dashboard updated** → Metrics reflect all changes
**Assertions:** Each step completes without error. Data is consistent across API and frontend. Take screenshots at each step.
---
### Test 12.2: Complete Workflow Walkthrough — Education User
Same 13-step walkthrough but verify education-specific features:
- Email language is student-appropriate
- Academic term awareness in planning/messaging
- Registration hold mentions for severely overdue
- Payment plan references
---
### Test 12.3: Complete Workflow Walkthrough — Wholesale User
Same 13-step walkthrough but verify wholesale-specific features:
- PO number references in emails
- Volume/relationship-aware messaging
- Deduction dispute handling
- Higher volume invoice processing
---
### Test 12.4: Complete Workflow Walkthrough — Generic User
Same 13-step walkthrough as baseline:
- No industry-specific language (generic professional)
- All features work without industry specialization
- Default business rules applied correctly
---
### Test 12.5: Security Verification
```bash
# Test 1: Unauthenticated access denied
curl -s http://localhost:8000/api/automations/ar/workflow/status/
```
**Assert:** Returns 401 Unauthorized
```bash
# Test 2: Cross-user data isolation — user A can't see user B's data
curl -s http://localhost:8000/api/invoices/ \
-H "Authorization: Bearer " | python3 -c "
import json, sys
data = json.load(sys.stdin)
invoices = data.get('results', data.get('data', []))
for inv in invoices:
user = inv.get('user', inv.get('user_id', 'unknown'))
print(f'Invoice {inv.get(\"invoice_number\")}: user={user}')
"
```
**Assert:** All invoices belong to construction user — NONE from other test users.
```bash
# Test 3: Invalid token rejected
curl -s http://localhost:8000/api/automations/ar/workflow/status/ \
-H "Authorization: Bearer invalid.token.here"
```
**Assert:** Returns 401
```bash
# Test 4: Expired token rejected (if you have one from earlier that expired)
```
**Assert:** Returns 401 with "token_expired" or similar
---
### Test 12.6: UI/UX Verification — All Pages (Chrome MCP)
Navigate to EVERY key page and verify:
**Pages to check:**
1. `/dashboard` — KPIs, widgets, activity feed
2. `/invoices` — Invoice list with filters
3. `/customers` — Customer list
4. `/customers/` — Customer detail with communication history
5. `/automations/review-queue` — Review queue
6. `/settings/ai-automation` — AI automation settings
7. `/settings/notifications` — Notification settings
8. `/hub` — Communications hub
**For EACH page assert:**
- [ ] Page loads without blank screen
- [ ] No JavaScript console errors
- [ ] All data is real (not mock/placeholder)
- [ ] No "undefined", "null", or "NaN" displayed
- [ ] No lorem ipsum or placeholder text
- [ ] All buttons are clickable and functional
- [ ] Loading states work (spinner shown while loading)
- [ ] Error states display properly (if API fails)
- [ ] Layout is not broken (no overlapping elements)
- [ ] Take screenshot for evidence
---
### Test 12.7: Performance Verification
```bash
# Test API response times
for endpoint in \
"/api/dashboard/" \
"/api/invoices/" \
"/api/automations/ar/action-queue/" \
"/api/automations/ar/activity-feed/" \
"/api/automations/scheduled/" \
"/api/automations/ar/business-rules/" \
"/api/notifications/"; do
time=$(curl -s -o /dev/null -w "%{time_total}" "http://localhost:8000$endpoint" \
-H "Authorization: Bearer ")
echo "$endpoint: ${time}s"
if (( $(echo "$time > 2.0" | bc -l) )); then
echo " *** SLOW: Over 2 second threshold ***"
fi
done
```
**Assertions:**
- ALL API endpoints respond in < 2 seconds
- Dashboard page loads in < 3 seconds (Chrome MCP timing)
- No endpoint takes > 5 seconds (critical failure)
---
### Test 12.8: Final Completeness Checklist
Run this final verification:
```bash
cd singoa-api && python -c "
import django, os; os.environ.setdefault('DJANGO_SETTINGS_MODULE','singoa_project.settings'); django.setup()
from authentication.models import User
from invoices.models import Customer, Invoice
from automations.models import PlannedAction, AutomationLog, ActionQueueItem, ActivityFeedItem, ImmutableAuditLog, BusinessRuleSet
test_emails = ['artest.construction@singoa.com', 'artest.education@singoa.com', 'artest.wholesale@singoa.com', 'artest.generic@singoa.com']
users = User.objects.filter(email__in=test_emails)
print('=== FINAL VERIFICATION ===')
print()
checks_passed = 0
checks_total = 0
for u in users:
print(f'--- {u.email} ---')
# Business rules exist
checks_total += 1
rules = BusinessRuleSet.objects.filter(user=u).exists()
print(f' Business rules: {\"PASS\" if rules else \"FAIL\"}')
if rules: checks_passed += 1
# Has customers
checks_total += 1
customers = Customer.objects.filter(user=u).count()
ok = customers >= 8
print(f' Customers: {customers} ({\"PASS\" if ok else \"FAIL - need 8\"})')
if ok: checks_passed += 1
# Has invoices
checks_total += 1
invoices = Invoice.objects.filter(user=u).count()
ok = invoices >= 15
print(f' Invoices: {invoices} ({\"PASS\" if ok else \"FAIL - need 15\"})')
if ok: checks_passed += 1
# Has automation logs (emails were sent)
checks_total += 1
logs = AutomationLog.objects.filter(user=u, status='SENT').count()
ok = logs >= 1
print(f' Emails sent: {logs} ({\"PASS\" if ok else \"FAIL - need >=1\"})')
if ok: checks_passed += 1
# Has activity feed entries
checks_total += 1
activity = ActivityFeedItem.objects.filter(user=u).count()
ok = activity >= 1
print(f' Activity feed: {activity} ({\"PASS\" if ok else \"FAIL\"})')
if ok: checks_passed += 1
# Has audit log entries
checks_total += 1
audit = ImmutableAuditLog.objects.filter(actor_id=u.id).count()
ok = audit >= 1
print(f' Audit log: {audit} ({\"PASS\" if ok else \"FAIL\"})')
if ok: checks_passed += 1
print()
print(f'=== RESULT: {checks_passed}/{checks_total} checks passed ===')
if checks_passed == checks_total:
print('ALL CHECKS PASSED!')
else:
print(f'FAILURES: {checks_total - checks_passed} checks failed')
"
```
**This is the FINAL gate. ALL checks must pass before the completion promise can be output.**
---
## PHASE 13: EMAIL INTEGRATION & REAL DELIVERY
This phase tests the entire email integration pipeline — Gmail/Outlook OAuth, token management, real email delivery to shivambindal155@gmail.com, and email sync.
### Test 13.1: Check Email Integration Status
```bash
curl -s http://localhost:8000/api/integrations/ \
-H "Authorization: Bearer $CONSTRUCTION_TOKEN" | python3 -c "
import sys,json; data=json.load(sys.stdin)
print(json.dumps(data, indent=2))
# Look for any email integrations (Gmail, Outlook, SendGrid)
"
```
**Assertions:**
- Response 200
- Note which email integrations are available
- If NO email integration exists, check settings.py EMAIL_BACKEND
- In dev mode, console backend is acceptable BUT real delivery test (13.5) must use actual service
---
### Test 13.2: Gmail OAuth Integration — Connect Account
```bash
# Check if Gmail OAuth is configured
cd singoa-api && python -c "
import django, os; os.environ.setdefault('DJANGO_SETTINGS_MODULE','singoa_project.settings'); django.setup()
from django.conf import settings
print('GOOGLE_CLIENT_ID:', 'SET' if settings.GOOGLE_CLIENT_ID else 'NOT SET')
print('GOOGLE_CLIENT_SECRET:', 'SET' if settings.GOOGLE_CLIENT_SECRET else 'NOT SET')
print('GOOGLE_REDIRECT_URI:', settings.GOOGLE_REDIRECT_URI)
from integrations.models import Integration
gmail_integrations = Integration.objects.filter(provider='gmail')
print(f'Existing Gmail integrations: {gmail_integrations.count()}')
for g in gmail_integrations:
print(f' User: {g.user.email}, Status: {g.status}, Connected: {g.is_connected}')
"
```
**Assertions:**
- Google OAuth credentials are configured (or note if not available for testing)
- If Gmail integrations exist, verify their status
- If tokens are expired, verify the refresh mechanism works
---
### Test 13.3: Email Service Provider Verification
```bash
cd singoa-api && python -c "
import django, os; os.environ.setdefault('DJANGO_SETTINGS_MODULE','singoa_project.settings'); django.setup()
from django.conf import settings
print('EMAIL_BACKEND:', settings.EMAIL_BACKEND)
print('EMAIL_HOST:', settings.EMAIL_HOST)
print('DEFAULT_FROM_EMAIL:', settings.DEFAULT_FROM_EMAIL)
print()
# Check SendGrid
sendgrid_key = os.getenv('SENDGRID_API_KEY', '')
print('SENDGRID_API_KEY:', 'SET' if sendgrid_key else 'NOT SET')
print()
# Check what happens when we try to send
from django.core.mail import send_mail
try:
result = send_mail(
'Test Subject - Singoa AR Test',
'This is a test email from Singoa AR Workflow E2E testing.',
settings.DEFAULT_FROM_EMAIL,
['shivambindal155@gmail.com'],
fail_silently=False,
)
print(f'send_mail result: {result}')
except Exception as e:
print(f'send_mail error: {type(e).__name__}: {e}')
"
```
**Assertions:**
- Email backend is identified (SendGrid, console, SMTP)
- If console backend: emails appear in Django console output
- If SendGrid: API key is configured and send succeeds
- Document the email delivery method for subsequent tests
---
### Test 13.4: Send Test Email via Gmail API Service
```bash
cd singoa-api && python -c "
import django, os; os.environ.setdefault('DJANGO_SETTINGS_MODULE','singoa_project.settings'); django.setup()
from integrations.models import Integration
from integrations.services.email_service import GmailService
# Find a Gmail integration
gmail = Integration.objects.filter(provider='gmail', status='active').first()
if gmail:
print(f'Using Gmail integration for: {gmail.user.email}')
service = GmailService(gmail.get_access_token())
result = service.send_email(
to='shivambindal155@gmail.com',
subject='[SINGOA TEST] Gmail API Direct Send Test',
body='This is a test email sent via Gmail API from Singoa AR E2E testing.'
)
print(f'Result: {result}')
else:
print('No active Gmail integration found.')
print('Testing via Django send_mail instead...')
from django.core.mail import send_mail
send_mail(
'[SINGOA TEST] Django Mail Test',
'This is a test email from Singoa AR E2E testing via Django mail.',
'noreply@singoa.com',
['shivambindal155@gmail.com'],
)
print('Email sent via Django mail backend')
"
```
**Assertions:**
- Email is actually sent (check shivambindal155@gmail.com inbox)
- If Gmail API: verify API call succeeds
- If Django mail: verify console output shows email content
- Note which method works for real delivery
---
### Test 13.5: REAL Email Delivery to shivambindal155@gmail.com via AR Workflow
This is the most important email test — trigger the actual AR workflow for the NovaTech customer (which uses shivambindal155@gmail.com) and verify a REAL email arrives.
```bash
# First, find the NovaTech customer and an overdue invoice
cd singoa-api && python -c "
import django, os; os.environ.setdefault('DJANGO_SETTINGS_MODULE','singoa_project.settings'); django.setup()
from invoices.models import Customer, Invoice
from authentication.models import User
# Find NovaTech across all test users
for email in ['artest.construction@singoa.com', 'artest.education@singoa.com', 'artest.wholesale@singoa.com', 'artest.generic@singoa.com']:
try:
user = User.objects.get(email=email)
nova = Customer.objects.filter(user=user, customer_email='shivambindal155@gmail.com').first()
if nova:
print(f'Found NovaTech: {nova.customer_name} (ID: {nova.id}) under {email}')
invoices = Invoice.objects.filter(customer=nova, status__in=['overdue', 'sent']).order_by('due_date')
for inv in invoices[:5]:
print(f' Invoice: {inv.invoice_number} | \${inv.total_amount} | Due: {inv.due_date} | Status: {inv.status}')
except User.DoesNotExist:
pass
"
```
Then trigger a reminder for NovaTech's overdue invoice:
```bash
curl -s -X POST http://localhost:8000/api/automations/ar/invoice//reminder/ \
-H "Authorization: Bearer $TOKEN" \
-H "Content-Type: application/json" | python3 -c "import sys,json; print(json.dumps(json.load(sys.stdin), indent=2))"
```
**Assertions:**
- Reminder API returns 200/201
- Check shivambindal155@gmail.com inbox for the ACTUAL email
- Email subject references the invoice number/amount
- Email body is personalized (not generic template)
- Email is from the expected sender address
- If email doesn't arrive: investigate email backend, check logs, fix the delivery pipeline
---
### Test 13.6: Email Sync — Fetch Inbound Emails
```bash
curl -s -X POST http://localhost:8000/api/integrations/emails/sync/ \
-H "Authorization: Bearer $CONSTRUCTION_TOKEN" \
-H "Content-Type: application/json" \
-d '{"days_back": 7}' | python3 -c "import sys,json; print(json.dumps(json.load(sys.stdin), indent=2))"
```
**Assertions:**
- Response 200
- Reports how many emails synced
- No errors in sync process
- Synced emails appear in communications hub
---
### Test 13.7: OAuth Token Refresh Mechanism
```bash
cd singoa-api && python -c "
import django, os; os.environ.setdefault('DJANGO_SETTINGS_MODULE','singoa_project.settings'); django.setup()
from integrations.models import Integration
from datetime import datetime, timedelta
from django.utils import timezone
# Check token expiry for all integrations
for intg in Integration.objects.filter(provider__in=['gmail', 'outlook']):
print(f'Provider: {intg.provider}, User: {intg.user.email}')
print(f' Status: {intg.status}')
print(f' Token expires: {intg.token_expires_at}')
if intg.token_expires_at:
remaining = intg.token_expires_at - timezone.now()
print(f' Time remaining: {remaining}')
if remaining.total_seconds() < 3600:
print(' WARNING: Token expires in < 1 hour')
print(f' Refresh token exists: {bool(intg.refresh_token)}')
print()
"
```
**Assertions:**
- Token expiry times are tracked
- Refresh tokens exist for OAuth integrations
- Proactive refresh task is scheduled in Celery beat
- Expired tokens trigger automatic refresh (not user intervention)
---
### Test 13.8: Email Delivery Failure Handling
```bash
# Test with an invalid email to verify error handling
cd singoa-api && python -c "
import django, os; os.environ.setdefault('DJANGO_SETTINGS_MODULE','singoa_project.settings'); django.setup()
from invoices.models import Customer
from authentication.models import User
# Temporarily set a customer email to invalid
user = User.objects.get(email='artest.generic@singoa.com')
customer = Customer.objects.filter(user=user).first()
original_email = customer.customer_email
customer.customer_email = 'invalid-email-address'
customer.save()
print(f'Set {customer.customer_name} email to: {customer.customer_email}')
print(f'Now trigger a reminder and verify it fails gracefully...')
print(f'Original email (restore after test): {original_email}')
"
```
Then try to send:
```bash
curl -s -X POST http://localhost:8000/api/automations/send-customer-email/ \
-H "Authorization: Bearer $GENERIC_TOKEN" \
-H "Content-Type: application/json" \
-d '{"customer_id": "", "subject": "Test", "body": "Test"}' | python3 -c "
import sys,json; data=json.load(sys.stdin); print(json.dumps(data, indent=2))"
```
Then restore:
```bash
cd singoa-api && python -c "
import django, os; os.environ.setdefault('DJANGO_SETTINGS_MODULE','singoa_project.settings'); django.setup()
from invoices.models import Customer
from authentication.models import User
user = User.objects.get(email='artest.generic@singoa.com')
customer = Customer.objects.filter(user=user).first()
customer.customer_email = ''
customer.save()
print(f'Restored email: {customer.customer_email}')
"
```
**Assertions:**
- Send attempt fails gracefully (not 500 error)
- Error message explains the issue (invalid email)
- ActionQueueItem created for manual review
- No crash or unhandled exception
- Email restored after test
---
### Test 13.9: Email Rate Limiting per Customer
```bash
# Send 5 rapid emails to same customer — verify rate limiting kicks in
for i in $(seq 1 5); do
echo "--- Attempt $i ---"
curl -s -X POST http://localhost:8000/api/automations/send-customer-email/ \
-H "Authorization: Bearer $CONSTRUCTION_TOKEN" \
-H "Content-Type: application/json" \
-d '{"customer_id": "", "subject": "Test $i", "body": "Rate limit test $i", "email_type": "reminder"}' | python3 -c "import sys,json; d=json.load(sys.stdin); print(d.get('status',''), d.get('error',''), d.get('message',''))"
sleep 1
done
```
**Assertions:**
- First 1-3 emails succeed (depending on max_emails_per_day setting)
- Subsequent emails are rate-limited with clear error message
- Rate limit is per-customer, not global
- Rate limit respects BusinessRuleSet.max_emails_per_day
---
### Test 13.10: Email Thread Tracking
```bash
# After sending a reminder, verify the email thread is tracked
cd singoa-api && python -c "
import django, os; os.environ.setdefault('DJANGO_SETTINGS_MODULE','singoa_project.settings'); django.setup()
from automations.models import AutomationLog
from authentication.models import User
user = User.objects.get(email='artest.construction@singoa.com')
logs = AutomationLog.objects.filter(user=user, status='SENT').order_by('-created_at')[:5]
for log in logs:
print(f'Log ID: {log.id}')
print(f' Customer: {log.customer.customer_name if log.customer else \"N/A\"}')
print(f' Invoice: {log.invoice.invoice_number if log.invoice else \"N/A\"}')
print(f' Action Type: {log.action_type}')
print(f' Details: {log.details}')
print(f' Created: {log.created_at}')
print()
"
```
**Assertions:**
- Each sent email has a corresponding AutomationLog
- Log includes customer reference, invoice reference, action type
- Details contain email subject/preview
- Chronological ordering is correct
---
### Test 13.11: Bounced Email Detection
```bash
# Simulate a bounced email for Zenith Global Trading (known bounced email in seed data)
cd singoa-api && python -c "
import django, os; os.environ.setdefault('DJANGO_SETTINGS_MODULE','singoa_project.settings'); django.setup()
from automations.models import AutomationLog, ActionQueueItem
from invoices.models import Customer
from authentication.models import User
user = User.objects.get(email='artest.construction@singoa.com')
zenith = Customer.objects.filter(user=user, customer_name__icontains='Zenith').first()
if zenith:
print(f'Zenith customer: {zenith.customer_name} ({zenith.customer_email})')
# Check if there are bounced logs
bounced = AutomationLog.objects.filter(customer=zenith, status='BOUNCED')
print(f'Bounced logs: {bounced.count()}')
# Check if action queue item was created for bounced email
queue = ActionQueueItem.objects.filter(customer=zenith, action_type__icontains='email')
print(f'Queue items for Zenith: {queue.count()}')
for q in queue:
print(f' Type: {q.action_type}, Status: {q.status}, Notes: {q.notes[:100] if q.notes else \"\"}')
"
```
**Assertions:**
- Bounced emails from seed data are tracked
- ActionQueueItem exists for bounced email customers
- System doesn't keep trying to send to bounced addresses
- Email resolution service flags invalid emails
---
### Test 13.12: Email Template Management
```bash
# List email templates
curl -s http://localhost:8000/api/automations/email-templates/ \
-H "Authorization: Bearer $CONSTRUCTION_TOKEN" | python3 -c "
import sys,json; data=json.load(sys.stdin)
if isinstance(data, list):
print(f'Templates found: {len(data)}')
for t in data[:5]:
print(f' - {t.get(\"name\",\"\")} | Type: {t.get(\"template_type\",\"\")} | Industry: {t.get(\"industry\",\"\")}')
elif isinstance(data, dict) and 'results' in data:
print(f'Templates found: {len(data[\"results\"])}')
for t in data['results'][:5]:
print(f' - {t.get(\"name\",\"\")} | Type: {t.get(\"template_type\",\"\")} | Industry: {t.get(\"industry\",\"\")}')
else:
print(json.dumps(data, indent=2))
"
```
**Assertions:**
- Template endpoint responds (200)
- Templates exist for different email types (reminder, follow-up, escalation, final notice)
- Templates have industry-specific variants if applicable
- Templates include merge fields/variables
---
### Test 13.13: Template Preview with Real Customer Data
```bash
curl -s -X POST http://localhost:8000/api/automations/template-preview/ \
-H "Authorization: Bearer $CONSTRUCTION_TOKEN" \
-H "Content-Type: application/json" \
-d '{
"template_id": "",
"customer_id": "",
"invoice_id": ""
}' | python3 -c "import sys,json; print(json.dumps(json.load(sys.stdin), indent=2))"
```
**Assertions:**
- Preview renders with real customer name, invoice amount, due date
- No unresolved {{variables}} or placeholders
- Professional formatting
- Industry-appropriate language
---
### Test 13.14: Email Open/Click Tracking (If Implemented)
```bash
cd singoa-api && python -c "
import django, os; os.environ.setdefault('DJANGO_SETTINGS_MODULE','singoa_project.settings'); django.setup()
from automations.models import AutomationLog, CommunicationEffectivenessLog
from authentication.models import User
# Check if effectiveness tracking exists
user = User.objects.get(email='artest.construction@singoa.com')
effectiveness = CommunicationEffectivenessLog.objects.filter(user=user)
print(f'Effectiveness logs: {effectiveness.count()}')
for e in effectiveness[:5]:
print(f' Tone: {e.tone}, Segment: {e.customer_segment}')
print(f' Opened: {e.opened}, Replied: {e.replied}, Paid: {e.resulted_in_payment}')
print()
"
```
**Assertions:**
- CommunicationEffectivenessLog model exists and is tracked
- After sending emails, effectiveness entries are created
- Fields track open rates, reply rates, payment conversion
---
### Test 13.15: Multi-Channel Delivery Preference
```bash
# Check if customer has channel preferences (email vs SMS vs slack)
cd singoa-api && python -c "
import django, os; os.environ.setdefault('DJANGO_SETTINGS_MODULE','singoa_project.settings'); django.setup()
from invoices.models import Customer
from authentication.models import User
user = User.objects.get(email='artest.construction@singoa.com')
customers = Customer.objects.filter(user=user)[:5]
for c in customers:
print(f'{c.customer_name}:')
print(f' Email: {c.customer_email}')
print(f' Phone: {c.phone}')
# Check for communication preferences
prefs = getattr(c, 'communication_preferences', None) or getattr(c, 'preferred_channel', None)
print(f' Preferred channel: {prefs or \"default (email)\"}')
print()
"
```
**Assertions:**
- Customer model supports communication channel preference
- Default channel is email
- System respects channel preference when sending
**Phase 13 is PASS when ALL 15 tests pass. Update state tracker.**
---
## PHASE 14: AI AGENT & CONVERSATIONS
This phase tests the AI agent system — the conversational interface where users can ask questions, get insights, and manage their AR workflow through natural language.
### Test 14.1: List AI Agent Conversations
```bash
curl -s http://localhost:8000/api/automations/agent/conversations/ \
-H "Authorization: Bearer $CONSTRUCTION_TOKEN" | python3 -c "
import sys,json; data=json.load(sys.stdin)
print(json.dumps(data, indent=2)[:2000])
"
```
**Assertions:**
- Response 200
- Returns list of conversations (may be empty initially)
- Each conversation has: id, title, created_at, message_count
---
### Test 14.2: Start New Agent Conversation
```bash
curl -s -X POST http://localhost:8000/api/automations/agent/chat/ \
-H "Authorization: Bearer $CONSTRUCTION_TOKEN" \
-H "Content-Type: application/json" \
-d '{
"message": "What are my most overdue invoices right now?",
"conversation_id": null
}' | python3 -c "import sys,json; print(json.dumps(json.load(sys.stdin), indent=2)[:3000])"
```
**Assertions:**
- Response 200
- Returns a conversation_id
- Agent response mentions actual invoice data (not generic/placeholder)
- Response references real customer names and amounts from seeded data
- Response is contextually relevant to the user's AR data
---
### Test 14.3: Continue Existing Conversation
```bash
curl -s -X POST http://localhost:8000/api/automations/agent/chat/ \
-H "Authorization: Bearer $CONSTRUCTION_TOKEN" \
-H "Content-Type: application/json" \
-d '{
"message": "Send a reminder to the most delinquent one",
"conversation_id": ""
}' | python3 -c "import sys,json; print(json.dumps(json.load(sys.stdin), indent=2)[:3000])"
```
**Assertions:**
- Agent understands context from previous message
- References the specific delinquent customer identified in 14.2
- Either executes the action or explains what needs approval
- Conversation history is maintained
---
### Test 14.4: Agent Context Awareness — Industry Specific
```bash
# Construction user asking about retainage
curl -s -X POST http://localhost:8000/api/automations/agent/chat/ \
-H "Authorization: Bearer $CONSTRUCTION_TOKEN" \
-H "Content-Type: application/json" \
-d '{"message": "Do any of my invoices have retainage holds?"}' | python3 -c "
import sys,json; print(json.dumps(json.load(sys.stdin), indent=2)[:3000])"
```
```bash
# Education user asking about academic terms
curl -s -X POST http://localhost:8000/api/automations/agent/chat/ \
-H "Authorization: Bearer $EDUCATION_TOKEN" \
-H "Content-Type: application/json" \
-d '{"message": "Show me overdue student accounts for this semester"}' | python3 -c "
import sys,json; print(json.dumps(json.load(sys.stdin), indent=2)[:3000])"
```
**Assertions:**
- Construction agent understands retainage, lien waivers, AIA billing
- Education agent understands semesters, student accounts, financial aid
- Responses use industry-appropriate terminology
- Data returned is filtered by the user's industry context
---
### Test 14.5: Agent Action Confirmation Flow
```bash
# Ask agent to do something that requires confirmation
curl -s -X POST http://localhost:8000/api/automations/agent/chat/ \
-H "Authorization: Bearer $CONSTRUCTION_TOKEN" \
-H "Content-Type: application/json" \
-d '{"message": "Write off the invoice for Phantom Enterprises — it is uncollectable"}' | python3 -c "
import sys,json; data=json.load(sys.stdin); print(json.dumps(data, indent=2)[:3000])"
```
**Assertions:**
- Agent does NOT immediately execute write-off
- Agent requests confirmation with details (customer name, amount, implications)
- Agent creates a pending approval or shows confirmation dialog
- Only after explicit confirm does the action execute
---
### Test 14.6: Agent Confirm Action
```bash
curl -s -X POST http://localhost:8000/api/automations/agent/confirm/ \
-H "Authorization: Bearer $CONSTRUCTION_TOKEN" \
-H "Content-Type: application/json" \
-d '{
"action_id": "",
"confirmed": true
}' | python3 -c "import sys,json; print(json.dumps(json.load(sys.stdin), indent=2))"
```
**Assertions:**
- Confirmation response acknowledges the action
- Action is executed (or queued for execution)
- Audit log entry created for the confirmed action
---
### Test 14.7: Agent Insights & Recommendations
```bash
curl -s http://localhost:8000/api/automations/agent/insights/ \
-H "Authorization: Bearer $CONSTRUCTION_TOKEN" | python3 -c "
import sys,json; data=json.load(sys.stdin); print(json.dumps(data, indent=2)[:3000])"
```
**Assertions:**
- Returns AI-generated insights about the user's AR data
- Insights are data-driven (reference actual numbers)
- Includes recommendations for next actions
- Industry-relevant insights
---
### Test 14.8: Agent Performance Metrics
```bash
curl -s http://localhost:8000/api/automations/agent/performance/ \
-H "Authorization: Bearer $CONSTRUCTION_TOKEN" | python3 -c "
import sys,json; print(json.dumps(json.load(sys.stdin), indent=2))"
```
**Assertions:**
- Returns agent performance stats
- Includes: total conversations, actions taken, accuracy, response time
- Metrics are realistic (not zeros or placeholders)
---
### Test 14.9: Agent Tools List
```bash
curl -s http://localhost:8000/api/automations/agent/tools/ \
-H "Authorization: Bearer $CONSTRUCTION_TOKEN" | python3 -c "
import sys,json; data=json.load(sys.stdin); print(json.dumps(data, indent=2)[:2000])"
```
**Assertions:**
- Returns list of tools the agent can use
- Tools include: send_reminder, check_invoice, create_payment_plan, write_off, etc.
- Each tool has description and required parameters
---
### Test 14.10: Agent Memory & Preferences
```bash
# Get AI preferences
curl -s http://localhost:8000/api/automations/ai/preferences/ \
-H "Authorization: Bearer $CONSTRUCTION_TOKEN" | python3 -c "
import sys,json; print(json.dumps(json.load(sys.stdin), indent=2))"
# Get AI context
curl -s http://localhost:8000/api/automations/ai/context/ \
-H "Authorization: Bearer $CONSTRUCTION_TOKEN" | python3 -c "
import sys,json; print(json.dumps(json.load(sys.stdin), indent=2)[:2000])"
```
**Assertions:**
- Preferences endpoint returns user's AI settings
- Context endpoint returns current AI context (customer data summary, recent actions)
- Context is accurate and reflects actual database state
---
### Test 14.11: Agent Cross-User Isolation
```bash
# Construction user's agent should NOT see education user's data
curl -s -X POST http://localhost:8000/api/automations/agent/chat/ \
-H "Authorization: Bearer $CONSTRUCTION_TOKEN" \
-H "Content-Type: application/json" \
-d '{"message": "Show me all customers"}' | python3 -c "
import sys,json; data=json.load(sys.stdin)
response = data.get('response', data.get('message', str(data)))
# Check that education-only customers don't appear
print(response[:2000])
"
```
**Assertions:**
- Construction user only sees construction customers
- No data leakage between users
- Agent respects user isolation boundaries
---
### Test 14.12: Frontend — AI Chat Interface (Chrome MCP)
**Steps:**
1. Navigate to `/dashboard` or `/automations` (wherever AI chat is)
2. Look for AI chat widget/button
3. Click to open chat
4. Type a question and send
5. Verify response renders properly
6. Take screenshot
**Assertions:**
- Chat interface is accessible and functional
- Messages render with proper formatting
- Loading indicator shows during AI response
- Chat history persists across page refreshes
- No console errors
**Phase 14 is PASS when ALL 12 tests pass. Update state tracker.**
---
## PHASE 15: LEARNING, SUGGESTIONS & A/B EXPERIMENTS
This phase tests the AI learning system — how the system learns from user behavior, generates suggestions, and runs A/B experiments to optimize email effectiveness.
### Test 15.1: Generate AI Suggestions
```bash
curl -s -X POST http://localhost:8000/api/automations/suggestions/generate/ \
-H "Authorization: Bearer $CONSTRUCTION_TOKEN" \
-H "Content-Type: application/json" | python3 -c "
import sys,json; data=json.load(sys.stdin); print(json.dumps(data, indent=2)[:3000])"
```
**Assertions:**
- Response 200/201
- Suggestions are generated based on user's actual data
- Each suggestion has: id, type, description, confidence, reasoning
---
### Test 15.2: List Suggestions
```bash
curl -s http://localhost:8000/api/automations/suggestions/ \
-H "Authorization: Bearer $CONSTRUCTION_TOKEN" | python3 -c "
import sys,json; data=json.load(sys.stdin); print(json.dumps(data, indent=2)[:3000])"
```
**Assertions:**
- Returns list of suggestions
- Suggestions are relevant to the user's AR data
- Different types: tone adjustment, timing change, escalation recommendation
---
### Test 15.3: Accept Suggestion
```bash
curl -s -X POST http://localhost:8000/api/automations/suggestions//accept/ \
-H "Authorization: Bearer $CONSTRUCTION_TOKEN" \
-H "Content-Type: application/json" | python3 -c "
import sys,json; print(json.dumps(json.load(sys.stdin), indent=2))"
```
**Assertions:**
- Suggestion marked as accepted
- Related settings/rules updated based on suggestion
- Activity feed entry created
- Audit log entry created
---
### Test 15.4: Dismiss Suggestion
```bash
curl -s -X POST http://localhost:8000/api/automations/suggestions//dismiss/ \
-H "Authorization: Bearer $CONSTRUCTION_TOKEN" \
-H "Content-Type: application/json" \
-d '{"reason": "Not applicable to my business"}' | python3 -c "
import sys,json; print(json.dumps(json.load(sys.stdin), indent=2))"
```
**Assertions:**
- Suggestion marked as dismissed
- Dismissal reason stored
- System learns from dismissal (future suggestions adjusted)
---
### Test 15.5: Approval Patterns Analysis
```bash
curl -s http://localhost:8000/api/automations/approval-patterns/ \
-H "Authorization: Bearer $CONSTRUCTION_TOKEN" | python3 -c "
import sys,json; print(json.dumps(json.load(sys.stdin), indent=2)[:2000])"
```
**Assertions:**
- Returns approval pattern analysis
- Shows which action types user typically approves/rejects
- Includes confidence scores for auto-approval recommendations
---
### Test 15.6: Communication Effectiveness Metrics
```bash
curl -s http://localhost:8000/api/automations/communication-effectiveness/ \
-H "Authorization: Bearer $CONSTRUCTION_TOKEN" | python3 -c "
import sys,json; print(json.dumps(json.load(sys.stdin), indent=2)[:2000])"
```
**Assertions:**
- Returns effectiveness data by tone, segment, industry
- Metrics include: open rate, reply rate, payment conversion rate
- Data reflects actual email sending results
---
### Test 15.7: Start A/B Experiment
```bash
curl -s -X POST http://localhost:8000/api/automations/experiments/ \
-H "Authorization: Bearer $CONSTRUCTION_TOKEN" \
-H "Content-Type: application/json" \
-d '{
"name": "Tone Test - Professional vs Friendly for VIP",
"description": "Test whether professional or friendly tone works better for VIP customers",
"variant_a": {"tone": "professional", "segment": "VIP"},
"variant_b": {"tone": "friendly", "segment": "VIP"},
"target_sample_size": 20,
"metric": "payment_rate"
}' | python3 -c "import sys,json; print(json.dumps(json.load(sys.stdin), indent=2))"
```
**Assertions:**
- Experiment created with status "draft" or "running"
- Returns experiment ID
- Both variants are properly configured
---
### Test 15.8: Get Experiment Details
```bash
curl -s http://localhost:8000/api/automations/experiments// \
-H "Authorization: Bearer $CONSTRUCTION_TOKEN" | python3 -c "
import sys,json; print(json.dumps(json.load(sys.stdin), indent=2))"
```
**Assertions:**
- Returns full experiment details
- Includes variant definitions, current progress, status
- Sample sizes are tracked
---
### Test 15.9: Experiment Results
```bash
curl -s http://localhost:8000/api/automations/experiments//results/ \
-H "Authorization: Bearer $CONSTRUCTION_TOKEN" | python3 -c "
import sys,json; print(json.dumps(json.load(sys.stdin), indent=2))"
```
**Assertions:**
- Returns results (may be preliminary if not enough data)
- Shows performance metrics for each variant
- Statistical significance indicator (if enough data)
---
### Test 15.10: Pause & Cancel Experiment
```bash
# Pause
curl -s -X POST http://localhost:8000/api/automations/experiments//pause/ \
-H "Authorization: Bearer $CONSTRUCTION_TOKEN" | python3 -c "
import sys,json; print(json.dumps(json.load(sys.stdin), indent=2))"
# Cancel
curl -s -X POST http://localhost:8000/api/automations/experiments//cancel/ \
-H "Authorization: Bearer $CONSTRUCTION_TOKEN" | python3 -c "
import sys,json; print(json.dumps(json.load(sys.stdin), indent=2))"
```
**Assertions:**
- Pause: experiment status changes to "paused"
- Cancel: experiment status changes to "cancelled"
- No more emails sent under cancelled experiment
---
### Test 15.11: Daily AI Insights Generation (Celery Task)
```bash
cd singoa-api && python -c "
import django, os; os.environ.setdefault('DJANGO_SETTINGS_MODULE','singoa_project.settings'); django.setup()
from automations.tasks_learning import generate_daily_ai_insights
result = generate_daily_ai_insights()
print(f'Result: {result}')
"
```
**Assertions:**
- Task runs without error
- Generates insights for all active users
- Insights are based on actual data trends
---
### Test 15.12: Learning Digest Email (Celery Task)
```bash
cd singoa-api && python -c "
import django, os; os.environ.setdefault('DJANGO_SETTINGS_MODULE','singoa_project.settings'); django.setup()
from automations.tasks_learning import send_learning_digest
result = send_learning_digest()
print(f'Result: {result}')
"
```
**Assertions:**
- Task runs without error
- Digest email generated for users with activity
- Digest content includes: actions taken, suggestions, performance metrics
**Phase 15 is PASS when ALL 12 tests pass. Update state tracker.**
---
## PHASE 16: ADVANCED AUTHORIZATION & AI BOUNDARIES
This phase tests the AI action boundary system — limits on what the AI can do autonomously vs what requires human approval.
### Test 16.1: List AI Boundaries
```bash
curl -s http://localhost:8000/api/automations/boundaries/ \
-H "Authorization: Bearer $CONSTRUCTION_TOKEN" | python3 -c "
import sys,json; data=json.load(sys.stdin); print(json.dumps(data, indent=2)[:2000])"
```
**Assertions:**
- Response 200
- Returns list of boundaries (may include defaults)
- Each boundary has: type, threshold, action_on_violation
---
### Test 16.2: Create AI Boundary — Dollar Limit
```bash
curl -s -X POST http://localhost:8000/api/automations/boundaries/ \
-H "Authorization: Bearer $CONSTRUCTION_TOKEN" \
-H "Content-Type: application/json" \
-d '{
"name": "Max Auto-Send Amount",
"description": "Require approval for emails about invoices over $10,000",
"boundary_type": "amount_threshold",
"threshold_value": 10000,
"action_on_violation": "require_approval",
"applies_to": ["send_reminder", "send_email"]
}' | python3 -c "import sys,json; print(json.dumps(json.load(sys.stdin), indent=2))"
```
**Assertions:**
- Boundary created successfully
- Returns boundary ID
- Boundary is active immediately
---
### Test 16.3: Verify Boundary Enforcement — Below Threshold (Auto-Proceed)
```bash
# Find an invoice under $10K and trigger reminder
curl -s -X POST http://localhost:8000/api/automations/boundaries/check/ \
-H "Authorization: Bearer $CONSTRUCTION_TOKEN" \
-H "Content-Type: application/json" \
-d '{
"action_type": "send_reminder",
"invoice_id": "",
"amount": 5000
}' | python3 -c "import sys,json; print(json.dumps(json.load(sys.stdin), indent=2))"
```
**Assertions:**
- Response indicates action is ALLOWED (no boundary violation)
- No approval required for invoices under threshold
---
### Test 16.4: Verify Boundary Enforcement — Above Threshold (Require Approval)
```bash
# Check an invoice over $10K
curl -s -X POST http://localhost:8000/api/automations/boundaries/check/ \
-H "Authorization: Bearer $CONSTRUCTION_TOKEN" \
-H "Content-Type: application/json" \
-d '{
"action_type": "send_reminder",
"invoice_id": "",
"amount": 25000
}' | python3 -c "import sys,json; print(json.dumps(json.load(sys.stdin), indent=2))"
```
**Assertions:**
- Response indicates action is BLOCKED (boundary violation)
- Violation details include which boundary was triggered
- Approval request created automatically
---
### Test 16.5: Parse Boundary from Natural Language
```bash
curl -s -X POST http://localhost:8000/api/automations/boundaries/parse/ \
-H "Authorization: Bearer $CONSTRUCTION_TOKEN" \
-H "Content-Type: application/json" \
-d '{
"text": "Never send more than 3 emails per week to any customer"
}' | python3 -c "import sys,json; print(json.dumps(json.load(sys.stdin), indent=2))"
```
**Assertions:**
- AI parses the natural language into a structured boundary
- Extracted: type=frequency_limit, value=3, period=week, scope=per_customer
- Boundary can be saved from the parsed result
---
### Test 16.6: Industry Default Boundaries
```bash
curl -s http://localhost:8000/api/automations/boundaries/industry-defaults/?industry=construction \
-H "Authorization: Bearer $CONSTRUCTION_TOKEN" | python3 -c "
import sys,json; print(json.dumps(json.load(sys.stdin), indent=2))"
```
**Assertions:**
- Returns industry-specific default boundaries
- Construction: higher thresholds (large project invoices)
- Education: restrictions during academic breaks
- Wholesale: EDI-specific boundaries
---
### Test 16.7: Get Boundary Suggestions
```bash
curl -s http://localhost:8000/api/automations/boundaries/suggestions/ \
-H "Authorization: Bearer $CONSTRUCTION_TOKEN" | python3 -c "
import sys,json; print(json.dumps(json.load(sys.stdin), indent=2)[:2000])"
```
**Assertions:**
- Returns AI-suggested boundaries based on user's data
- Suggestions are relevant and reasonable
- Each suggestion explains the reasoning
---
### Test 16.8: Update Boundary
```bash
curl -s -X PUT http://localhost:8000/api/automations/boundaries// \
-H "Authorization: Bearer $CONSTRUCTION_TOKEN" \
-H "Content-Type: application/json" \
-d '{
"threshold_value": 15000,
"description": "Updated: Require approval for invoices over $15,000"
}' | python3 -c "import sys,json; print(json.dumps(json.load(sys.stdin), indent=2))"
```
**Assertions:**
- Boundary updated successfully
- New threshold takes effect immediately
- Audit log entry for the change
---
### Test 16.9: Delete Boundary
```bash
curl -s -X DELETE http://localhost:8000/api/automations/boundaries// \
-H "Authorization: Bearer $CONSTRUCTION_TOKEN" | python3 -c "
import sys,json
try:
data = json.load(sys.stdin)
print(json.dumps(data, indent=2))
except:
print('Boundary deleted (empty response)')
"
```
**Assertions:**
- Boundary deleted (204 or 200)
- Boundary no longer enforced
- Audit log entry for deletion
---
### Test 16.10: Cross-User Boundary Isolation
```bash
# Construction user's boundaries should not affect education user
curl -s http://localhost:8000/api/automations/boundaries/ \
-H "Authorization: Bearer $EDUCATION_TOKEN" | python3 -c "
import sys,json; data=json.load(sys.stdin)
print(f'Education user boundaries: {len(data) if isinstance(data, list) else data}')
# Should NOT contain construction user's custom boundaries
"
```
**Assertions:**
- Education user's boundary list doesn't include construction user's custom boundaries
- Each user has their own boundary space
- Default boundaries may be shared but customizations are isolated
**Phase 16 is PASS when ALL 10 tests pass. Update state tracker.**
---
## PHASE 17: ADVANCED APPROVAL WORKFLOWS
This phase tests the full approval system — approval requests, token-based approvals (from email links), deferral, escalation, and bulk operations.
### Test 17.1: List Approval Requests
```bash
curl -s http://localhost:8000/api/automations/approvals/ \
-H "Authorization: Bearer $CONSTRUCTION_TOKEN" | python3 -c "
import sys,json; data=json.load(sys.stdin); print(json.dumps(data, indent=2)[:2000])"
```
**Assertions:**
- Response 200
- Returns list of approval requests
- Each has: id, type, status, created_at, details
---
### Test 17.2: Create Approval Request (High-Value Action)
```bash
# Trigger an action that requires approval
curl -s -X POST http://localhost:8000/api/automations/ar/collection/campaign/ \
-H "Authorization: Bearer $CONSTRUCTION_TOKEN" \
-H "Content-Type: application/json" \
-d '{
"customer_ids": [""],
"action_type": "escalation",
"tone": "firm"
}' | python3 -c "import sys,json; print(json.dumps(json.load(sys.stdin), indent=2))"
```
**Assertions:**
- If action requires approval: approval request created with status "pending"
- If auto-approved: action proceeds with audit trail
- Response includes approval_request_id if approval needed
---
### Test 17.3: Approval Request Detail
```bash
curl -s http://localhost:8000/api/automations/approvals// \
-H "Authorization: Bearer $CONSTRUCTION_TOKEN" | python3 -c "
import sys,json; print(json.dumps(json.load(sys.stdin), indent=2))"
```
**Assertions:**
- Returns full approval details
- Includes: action details, customer info, invoice info, risk assessment
- Shows approval token (if email-based approval is enabled)
---
### Test 17.4: Approve via API
```bash
curl -s -X POST http://localhost:8000/api/automations/approvals//approve/ \
-H "Authorization: Bearer $CONSTRUCTION_TOKEN" \
-H "Content-Type: application/json" \
-d '{"notes": "Approved - customer has been unresponsive for 90+ days"}' | python3 -c "
import sys,json; print(json.dumps(json.load(sys.stdin), indent=2))"
```
**Assertions:**
- Approval status changes to "approved"
- Original action is executed
- Audit log entry with approver info and notes
- Notification sent confirming approval
---
### Test 17.5: Deny Approval
```bash
curl -s -X POST http://localhost:8000/api/automations/approvals//deny/ \
-H "Authorization: Bearer $CONSTRUCTION_TOKEN" \
-H "Content-Type: application/json" \
-d '{"reason": "Customer called — payment plan agreed"}' | python3 -c "
import sys,json; print(json.dumps(json.load(sys.stdin), indent=2))"
```
**Assertions:**
- Approval status changes to "denied"
- Original action is NOT executed
- Denial reason stored
- Audit log entry
---
### Test 17.6: Defer Approval
```bash
curl -s -X POST http://localhost:8000/api/automations/approvals//defer/ \
-H "Authorization: Bearer $CONSTRUCTION_TOKEN" \
-H "Content-Type: application/json" \
-d '{"defer_until": "2026-03-08", "reason": "Waiting for payment to clear"}' | python3 -c "
import sys,json; print(json.dumps(json.load(sys.stdin), indent=2))"
```
**Assertions:**
- Approval status changes to "deferred"
- Deferred until date is stored
- System will re-present the approval on the defer date
---
### Test 17.7: Token-Based Approval (Email Link)
```bash
# Get the approval token
cd singoa-api && python -c "
import django, os; os.environ.setdefault('DJANGO_SETTINGS_MODULE','singoa_project.settings'); django.setup()
from automations.models import ApprovalRequest
req = ApprovalRequest.objects.filter(status='pending').first()
if req:
print(f'Approval: {req.id}')
print(f'Token: {req.token}')
print(f'Action: {req.action_type}')
else:
print('No pending approvals found')
"
```
Then use the token:
```bash
curl -s -X POST http://localhost:8000/api/automations/approvals/token//respond/ \
-H "Content-Type: application/json" \
-d '{"action": "approve"}' | python3 -c "
import sys,json; print(json.dumps(json.load(sys.stdin), indent=2))"
```
**Assertions:**
- Token-based approval works without JWT auth (email link scenario)
- Approval is processed correctly
- Token becomes invalid after use (one-time use)
- Expired tokens are rejected
---
### Test 17.8: Approval Stats
```bash
curl -s http://localhost:8000/api/automations/approvals/stats/ \
-H "Authorization: Bearer $CONSTRUCTION_TOKEN" | python3 -c "
import sys,json; print(json.dumps(json.load(sys.stdin), indent=2))"
```
**Assertions:**
- Returns: total_pending, total_approved, total_denied, total_deferred
- Average response time
- Approval rate percentage
- Stats are accurate based on actual data
---
### Test 17.9: Expired Approval Cleanup (Celery Task)
```bash
cd singoa-api && python -c "
import django, os; os.environ.setdefault('DJANGO_SETTINGS_MODULE','singoa_project.settings'); django.setup()
from notifications.tasks import expire_pending_approvals
result = expire_pending_approvals()
print(f'Result: {result}')
"
```
**Assertions:**
- Task runs without error
- Old pending approvals (>7 days) are marked as expired
- Expired approvals don't block workflow
---
### Test 17.10: Review Queue — Unified View
```bash
curl -s http://localhost:8000/api/automations/review-queue/ \
-H "Authorization: Bearer $CONSTRUCTION_TOKEN" \
-H "Content-Type: application/json" | python3 -c "
import sys,json; data=json.load(sys.stdin); print(json.dumps(data, indent=2)[:3000])"
```
**Assertions:**
- Returns unified view of ALL pending items (action queue + approvals + PO reviews)
- Items sorted by priority/urgency
- Each item has enough context to make a decision
---
### Test 17.11: Bulk Approve — Multiple Items
```bash
curl -s -X POST http://localhost:8000/api/automations/review-queue/bulk-approve/ \
-H "Authorization: Bearer $CONSTRUCTION_TOKEN" \
-H "Content-Type: application/json" \
-d '{
"item_ids": ["", "", ""],
"notes": "Bulk approved — routine reminders"
}' | python3 -c "import sys,json; print(json.dumps(json.load(sys.stdin), indent=2))"
```
**Assertions:**
- All specified items approved
- Each item has individual audit log entry
- Response shows success/failure per item
- Failed items (if any) don't block successful ones
---
### Test 17.12: Auto-Approve Eligible Items
```bash
curl -s -X POST http://localhost:8000/api/automations/review-queue/auto-approve/ \
-H "Authorization: Bearer $CONSTRUCTION_TOKEN" \
-H "Content-Type: application/json" | python3 -c "
import sys,json; print(json.dumps(json.load(sys.stdin), indent=2))"
```
**Assertions:**
- Items meeting auto-approval criteria are approved
- Criteria based on: learned patterns, boundary compliance, low risk
- Items NOT meeting criteria remain pending
- Report of what was auto-approved and why
**Phase 17 is PASS when ALL 12 tests pass. Update state tracker.**
---
## PHASE 18: NATURAL LANGUAGE RULES & EMAIL TEMPLATES
### Test 18.1: Parse Natural Language Rule
```bash
curl -s -X POST http://localhost:8000/api/automations/rules/parse/ \
-H "Authorization: Bearer $CONSTRUCTION_TOKEN" \
-H "Content-Type: application/json" \
-d '{
"text": "If a customer has not paid within 45 days and their invoice is over $5000, escalate to a firm tone and send reminders every 3 days"
}' | python3 -c "import sys,json; print(json.dumps(json.load(sys.stdin), indent=2))"
```
**Assertions:**
- AI parses natural language into structured rule
- Extracted conditions: days_overdue > 45, amount > 5000
- Extracted actions: tone = firm, frequency = 3 days
- Parsed result is valid and can be saved
---
### Test 18.2: List Rule Templates
```bash
curl -s http://localhost:8000/api/automations/rules/templates/ \
-H "Authorization: Bearer $CONSTRUCTION_TOKEN" | python3 -c "
import sys,json; data=json.load(sys.stdin); print(json.dumps(data, indent=2)[:3000])"
```
**Assertions:**
- Returns predefined rule templates
- Templates cover common scenarios: escalation, frequency, tone adjustment
- Industry-specific templates available
---
### Test 18.3: Create Rule from Template
```bash
curl -s -X POST http://localhost:8000/api/automations/rules/from-template/ \
-H "Authorization: Bearer $CONSTRUCTION_TOKEN" \
-H "Content-Type: application/json" \
-d '{
"template_id": "",
"customizations": {
"days_overdue": 30,
"amount_threshold": 10000
}
}' | python3 -c "import sys,json; print(json.dumps(json.load(sys.stdin), indent=2))"
```
**Assertions:**
- Rule created from template with customizations applied
- Rule is in "active" or "draft" state
- Customized values override template defaults
---
### Test 18.4: Simulate Rule Behavior
```bash
curl -s -X POST http://localhost:8000/api/automations/rules/simulate/ \
-H "Authorization: Bearer $CONSTRUCTION_TOKEN" \
-H "Content-Type: application/json" \
-d '{
"rule_id": "",
"test_data": {
"customer_segment": "VIP",
"days_overdue": 50,
"amount": 15000,
"previous_reminders": 3
}
}' | python3 -c "import sys,json; print(json.dumps(json.load(sys.stdin), indent=2))"
```
**Assertions:**
- Simulation runs without modifying real data
- Returns: would_trigger (true/false), matched_conditions, resulting_actions
- Explains which conditions matched and what would happen
---
### Test 18.5: Activate/Deactivate Rule
```bash
# Activate
curl -s -X POST http://localhost:8000/api/automations/rules//activate/ \
-H "Authorization: Bearer $CONSTRUCTION_TOKEN" | python3 -c "
import sys,json; print(json.dumps(json.load(sys.stdin), indent=2))"
# Deactivate
curl -s -X POST http://localhost:8000/api/automations/rules//deactivate/ \
-H "Authorization: Bearer $CONSTRUCTION_TOKEN" | python3 -c "
import sys,json; print(json.dumps(json.load(sys.stdin), indent=2))"
```
**Assertions:**
- Rule status toggles correctly
- Active rules affect workflow decisions
- Deactivated rules don't trigger
---
### Test 18.6: Rule Chat — Create Rule via Conversation
```bash
curl -s -X POST http://localhost:8000/api/automations/rules/chat/ \
-H "Authorization: Bearer $CONSTRUCTION_TOKEN" \
-H "Content-Type: application/json" \
-d '{
"message": "I want to stop sending reminders to customers who have an active dispute"
}' | python3 -c "import sys,json; print(json.dumps(json.load(sys.stdin), indent=2)[:2000])"
```
**Assertions:**
- AI understands the intent
- Proposes a structured rule
- User can confirm to create the rule
- Rule is valid and enforceable
---
### Test 18.7: List Active NL Rules
```bash
curl -s http://localhost:8000/api/automations/rules/nl/ \
-H "Authorization: Bearer $CONSTRUCTION_TOKEN" | python3 -c "
import sys,json; data=json.load(sys.stdin); print(json.dumps(data, indent=2)[:2000])"
```
**Assertions:**
- Returns all user-created NL rules
- Each rule shows: original text, parsed conditions, status, created_at
- Rules are properly associated with the user
---
### Test 18.8: Create Email Template
```bash
curl -s -X POST http://localhost:8000/api/automations/email-templates/ \
-H "Authorization: Bearer $CONSTRUCTION_TOKEN" \
-H "Content-Type: application/json" \
-d '{
"name": "Construction Retainage Follow-Up",
"template_type": "follow_up",
"industry": "construction",
"subject": "Retainage Release Request - {{project_name}}",
"body": "Dear {{customer_name}},\n\nWe are writing regarding the retainage balance of {{amount}} for {{project_name}}. As the project milestone has been completed and approved, we kindly request the release of the retainage funds.\n\nPlease let us know if you need any additional documentation.\n\nBest regards"
}' | python3 -c "import sys,json; print(json.dumps(json.load(sys.stdin), indent=2))"
```
**Assertions:**
- Template created with all fields
- Template variables ({{customer_name}}, etc.) preserved
- Industry association correct
---
### Test 18.9: Update Email Template
```bash
curl -s -X PUT http://localhost:8000/api/automations/email-templates// \
-H "Authorization: Bearer $CONSTRUCTION_TOKEN" \
-H "Content-Type: application/json" \
-d '{
"subject": "Updated: Retainage Release - {{project_name}} ({{invoice_number}})"
}' | python3 -c "import sys,json; print(json.dumps(json.load(sys.stdin), indent=2))"
```
**Assertions:**
- Template updated, only specified fields changed
- Other fields preserved
- Updated_at timestamp changed
---
### Test 18.10: Delete Email Template
```bash
curl -s -X DELETE http://localhost:8000/api/automations/email-templates// \
-H "Authorization: Bearer $CONSTRUCTION_TOKEN"
echo "Status: $?"
```
**Assertions:**
- Template deleted (204)
- Template no longer appears in list
- Existing emails sent using this template are not affected
**Phase 18 is PASS when ALL 10 tests pass. Update state tracker.**
---
## PHASE 19: PRIVACY, SECURITY & PERMISSIONS
### Test 19.1: PII Data Masking — Verify Before AI Call
```bash
cd singoa-api && python -c "
import django, os; os.environ.setdefault('DJANGO_SETTINGS_MODULE','singoa_project.settings'); django.setup()
from automations.services.data_masking_service import DataMaskingService
service = DataMaskingService()
# Test masking
test_data = {
'customer_name': 'John Smith',
'customer_email': 'john.smith@example.com',
'phone': '+15551234567',
'address': '123 Main St, Anytown, USA',
'amount': 15000,
'invoice_number': 'INV-001'
}
masked = service.mask(test_data)
print('MASKED DATA:')
for k, v in masked.items():
print(f' {k}: {v}')
print()
# Verify PII is not visible
assert 'John Smith' not in str(masked), 'Name not masked!'
assert 'john.smith@example.com' not in str(masked), 'Email not masked!'
assert '+15551234567' not in str(masked), 'Phone not masked!'
assert '123 Main St' not in str(masked), 'Address not masked!'
print('ALL PII PROPERLY MASKED')
# Unmask
unmasked = service.unmask(masked)
print()
print('UNMASKED DATA:')
for k, v in unmasked.items():
print(f' {k}: {v}')
assert unmasked['customer_name'] == 'John Smith', 'Unmask failed!'
print('UNMASK VERIFIED')
"
```
**Assertions:**
- PII fields (name, email, phone, address) are masked with tokens (CUST_XXXXX, EMAIL_XXXXX)
- Non-PII fields (amount, invoice_number) are NOT masked
- Unmasking restores original values
- Tokens are deterministic (same input → same token)
---
### Test 19.2: Privacy Stats Endpoint
```bash
curl -s http://localhost:8000/api/automations/privacy/stats/ \
-H "Authorization: Bearer $CONSTRUCTION_TOKEN" | python3 -c "
import sys,json; print(json.dumps(json.load(sys.stdin), indent=2))"
```
**Assertions:**
- Returns privacy statistics
- Includes: total_masking_operations, masking_failures, data_types_masked
- All masking operations succeeded (0 failures)
---
### Test 19.3: Privacy Audit Log
```bash
curl -s http://localhost:8000/api/automations/privacy/audit-log/ \
-H "Authorization: Bearer $CONSTRUCTION_TOKEN" | python3 -c "
import sys,json; data=json.load(sys.stdin); print(json.dumps(data, indent=2)[:2000])"
```
**Assertions:**
- Audit log tracks all PII access events
- Each entry: who accessed, what data, when, why
- Log is immutable (entries can't be modified)
---
### Test 19.4: Cross-User Data Isolation — Invoices
```bash
# Education user tries to access construction user's invoice
cd singoa-api && python -c "
import django, os; os.environ.setdefault('DJANGO_SETTINGS_MODULE','singoa_project.settings'); django.setup()
from invoices.models import Invoice
from authentication.models import User
construction_user = User.objects.get(email='artest.construction@singoa.com')
education_user = User.objects.get(email='artest.education@singoa.com')
# Get a construction invoice ID
c_invoice = Invoice.objects.filter(user=construction_user).first()
print(f'Construction invoice: {c_invoice.id}')
# Try to access it as education user (via ORM, simulating API)
e_invoices = Invoice.objects.filter(user=education_user, id=c_invoice.id)
print(f'Education user can see it: {e_invoices.exists()}')
assert not e_invoices.exists(), 'SECURITY VIOLATION: Cross-user data access!'
print('PASS: Cross-user isolation verified')
"
```
**Assertions:**
- Education user cannot access construction user's invoices
- Query returns empty, not 404 or error (prevents enumeration)
---
### Test 19.5: Cross-User Data Isolation — API Level
```bash
# Get a construction user's invoice ID, then try with education token
CONSTRUCTION_INVOICE_ID=$(cd singoa-api && python -c "
import django, os; os.environ.setdefault('DJANGO_SETTINGS_MODULE','singoa_project.settings'); django.setup()
from invoices.models import Invoice
from authentication.models import User
u = User.objects.get(email='artest.construction@singoa.com')
inv = Invoice.objects.filter(user=u).first()
print(inv.id)
")
curl -s http://localhost:8000/api/invoices/$CONSTRUCTION_INVOICE_ID/ \
-H "Authorization: Bearer $EDUCATION_TOKEN" | python3 -c "
import sys,json; data=json.load(sys.stdin); print(json.dumps(data, indent=2))
# Should be 404 or 403, NOT the invoice data
"
```
**Assertions:**
- Returns 404 or 403 (not the invoice data)
- No data leakage in error response
- Access attempt is logged
---
### Test 19.6: Cross-User Data Isolation — Customers
```bash
curl -s http://localhost:8000/api/invoices/customers/ \
-H "Authorization: Bearer $CONSTRUCTION_TOKEN" | python3 -c "
import sys,json; data=json.load(sys.stdin)
customers = data if isinstance(data, list) else data.get('results', [])
emails = set()
for c in customers:
emails.add(c.get('customer_email',''))
print(f'Construction user sees {len(customers)} customers')
"
curl -s http://localhost:8000/api/invoices/customers/ \
-H "Authorization: Bearer $EDUCATION_TOKEN" | python3 -c "
import sys,json; data=json.load(sys.stdin)
customers = data if isinstance(data, list) else data.get('results', [])
print(f'Education user sees {len(customers)} customers')
"
```
**Assertions:**
- Each user sees ONLY their own customers
- No overlapping customer IDs between users
- Customer count matches seeded data
---
### Test 19.7: JWT Token Expiry & Refresh
```bash
# Get a fresh token pair
curl -s -X POST http://localhost:8000/api/auth/login/ \
-H "Content-Type: application/json" \
-d '{"email": "artest.construction@singoa.com", "password": "TestPass123!"}' | python3 -c "
import sys,json; data=json.load(sys.stdin)
print('Access token:', data.get('access','')[:50] + '...')
print('Refresh token:', data.get('refresh','')[:50] + '...')
# Decode the access token to check expiry
import base64
parts = data['access'].split('.')
payload = base64.urlsafe_b64decode(parts[1] + '==')
import json as j
p = j.loads(payload)
print(f'Token expires: {p.get(\"exp\",\"\")}')
print(f'User ID: {p.get(\"user_id\",\"\")}')
"
```
**Assertions:**
- Login returns both access and refresh tokens
- Access token has reasonable expiry (15-60 minutes)
- Token contains user_id claim
---
### Test 19.8: Expired Token Rejection
```bash
# Use a deliberately expired/invalid token
curl -s http://localhost:8000/api/invoices/ \
-H "Authorization: Bearer eyJhbGciOiJIUzI1NiIsInR5cCI6IkpXVCJ9.eyJ1c2VyX2lkIjoxLCJleHAiOjE1MDAwMDAwMDB9.invalid" | python3 -c "
import sys,json; data=json.load(sys.stdin); print(json.dumps(data, indent=2))"
```
**Assertions:**
- Returns 401 Unauthorized
- Error message is generic (doesn't reveal internal details)
- No data returned
---
### Test 19.9: Token Refresh Flow
```bash
curl -s -X POST http://localhost:8000/api/auth/token/refresh/ \
-H "Content-Type: application/json" \
-d "{\"refresh\": \"$CONSTRUCTION_REFRESH_TOKEN\"}" | python3 -c "
import sys,json; data=json.load(sys.stdin)
print('New access token:', data.get('access','')[:50] + '...' if data.get('access') else 'FAILED')
"
```
**Assertions:**
- Refresh returns a new access token
- New access token is valid for API calls
- Old access token may still work until expiry
---
### Test 19.10: Unauthenticated Access Rejection
```bash
# Try all major endpoints without auth
for endpoint in "invoices/" "invoices/customers/" "automations/ar/workflow/status/" "automations/ar/action-queue/" "communications/hub/messages/" "notifications/" "dashboard/"; do
STATUS=$(curl -s -o /dev/null -w "%{http_code}" http://localhost:8000/api/$endpoint)
echo "$endpoint -> $STATUS"
done
```
**Assertions:**
- ALL endpoints return 401 without authentication
- No endpoint leaks data to unauthenticated users
- Error responses are consistent
---
### Test 19.11: SQL Injection Prevention
```bash
curl -s "http://localhost:8000/api/invoices/customers/?search='; DROP TABLE invoices_customer; --" \
-H "Authorization: Bearer $CONSTRUCTION_TOKEN" | python3 -c "
import sys,json; data=json.load(sys.stdin)
print('Response received (DB not dropped):')
print(json.dumps(data, indent=2)[:500])
"
# Verify table still exists
cd singoa-api && python -c "
import django, os; os.environ.setdefault('DJANGO_SETTINGS_MODULE','singoa_project.settings'); django.setup()
from invoices.models import Customer
print(f'Customers still exist: {Customer.objects.count()}')
"
```
**Assertions:**
- Injection attempt doesn't crash the server
- Database tables are intact
- Query is properly parameterized
---
### Test 19.12: XSS Prevention — Input Sanitization
```bash
curl -s -X POST http://localhost:8000/api/invoices/customers/ \
-H "Authorization: Bearer $CONSTRUCTION_TOKEN" \
-H "Content-Type: application/json" \
-d '{
"customer_name": "",
"customer_email": "xss@test.com"
}' | python3 -c "
import sys,json; data=json.load(sys.stdin)
name = data.get('customer_name', '')
print(f'Stored name: {name}')
assert '