timebank-cc-public/references/translations/TRANSLATION_UPDATE_GUIDE.md

# Translation Update Guide

This guide explains how to update all translation files when new content with translation strings is added to views.

## Overview

The application uses Laravel's translation system with TWO types of translation files:

1. **JSON Translation Files** (for simple strings) - managed by `kargnas/laravel-ai-translator` package
2. **PHP Translation Files** (for dynamic strings with parameters) - managed manually

**JSON Translation Files Location:** `resources/lang/`
- `en.json` - English (source language)
- `nl.json` - Dutch
- `de.json` - German
- `es.json` - Spanish
- `fr.json` - French

**PHP Translation Files Location:** `resources/lang/[locale]/`
- `messages.php` - Dynamic messages with placeholders (e.g., `:name`, `:count`)
- `routes.php` - Route-related translations
- `validation.php` - Validation messages (Laravel default)
- `passwords.php` - Password reset messages (Laravel default)

**IMPORTANT:** PHP translation files should NEVER be converted to JSON format. Laravel automatically uses PHP files when the key is not found in JSON files. Keep them separate to avoid conflicts.

## When to Update Translations

Update translations whenever you:
- Add new `__('Translation string')` calls in Blade views
- Add new `__('Translation string')` calls in PHP code
- Add new `trans('Translation string')` calls
- Modify existing translation strings (creates new keys)

## Step-by-Step Update Process

### Step 1: Add English Translation Strings

First, add your new translation strings to `resources/lang/en.json`:

```bash
# Open the English translation file
nano resources/lang/en.json
```

Add your new keys in alphabetical order:

```json
{
  "existing.key": "Existing value",
  "new.key.one": "New translation string one",
  "new.key.two": "New translation string two",
  "another.key": "Another value"
}
```

**Important Rules for Translation Keys:**
- Use descriptive, meaningful key names
- Use dot notation for organization (e.g., `messages.welcome`, `buttons.submit`)
- Keep keys lowercase
- Use underscores for spaces in multi-word keys
- Avoid special characters except dots and underscores

### Step 2: Sync Translation Files

Sync all language files to ensure they have the same keys as `en.json`:

```bash
php artisan ai-translator:sync-json
```

This command:
- Adds missing keys from `en.json` to all other language files
- Removes keys that don't exist in `en.json` from other files
- Keeps existing translations intact
- Adds English values as placeholders for new keys

**Expected Output:**
```
Syncing translation files...
✓ nl.json: Added 15 keys, removed 0 keys
✓ de.json: Added 15 keys, removed 0 keys
✓ es.json: Added 15 keys, removed 0 keys
✓ fr.json: Added 15 keys, removed 0 keys
```

### Step 3: Translate New Keys with AI

Translate the new keys to all languages using the AI translator:

#### Option A: Translate All Languages Sequentially

Use the provided script to translate all languages in sequence:

```bash
./translate-new-keys.sh
```

#### Option B: Translate Each Language Individually

Translate Dutch:
```bash
php artisan ai-translator:translate-json --source=en --locale=nl --non-interactive --chunk=100
```

Translate German:
```bash
php artisan ai-translator:translate-json --source=en --locale=de --non-interactive --chunk=100
```

Translate Spanish:
```bash
php artisan ai-translator:translate-json --source=en --locale=es --non-interactive --chunk=100
```

Translate French:
```bash
php artisan ai-translator:translate-json --source=en --locale=fr --non-interactive --chunk=100
```

**Translation Parameters:**
- `--source=en` - Source language (English)
- `--locale=XX` - Target language code
- `--non-interactive` - Don't ask for confirmation
- `--chunk=100` - Process 100 keys at a time (reduces API load)

### Step 4: Verify Translation Results

Check that all files now have the same number of keys:

```bash
php -r "
\$en = json_decode(file_get_contents('resources/lang/en.json'), true);
\$nl = json_decode(file_get_contents('resources/lang/nl.json'), true);
\$de = json_decode(file_get_contents('resources/lang/de.json'), true);
\$es = json_decode(file_get_contents('resources/lang/es.json'), true);
\$fr = json_decode(file_get_contents('resources/lang/fr.json'), true);

echo \"Translation key counts:\n\";
echo \"  en: \" . count(\$en) . \" keys\n\";
echo \"  nl: \" . count(\$nl) . \" keys\n\";
echo \"  de: \" . count(\$de) . \" keys\n\";
echo \"  es: \" . count(\$es) . \" keys\n\";
echo \"  fr: \" . count(\$fr) . \" keys\n\";
"
```

**Expected Result:** All files should have the same number of keys.

### Step 5: Test Translations in Application

1. Clear Laravel cache:
```bash
php artisan config:clear
php artisan cache:clear
```

2. Test the new translations in your browser by switching languages
3. Verify that all new strings appear correctly translated

## Quick Reference Scripts

### translate-new-keys.sh

Create this script in your project root for easy sequential translation:

```bash
#!/bin/bash

echo "=== TRANSLATING NEW KEYS TO ALL LANGUAGES ==="
echo ""

# Sync first to ensure all files have the same keys
echo "Step 1: Syncing translation files..."
php artisan ai-translator:sync-json
echo ""

# Translate each language
echo "Step 2: Translating to Dutch (nl)..."
php artisan ai-translator:translate-json --source=en --locale=nl --non-interactive --chunk=100
sleep 5

echo ""
echo "Step 3: Translating to German (de)..."
php artisan ai-translator:translate-json --source=en --locale=de --non-interactive --chunk=100
sleep 5

echo ""
echo "Step 4: Translating to Spanish (es)..."
php artisan ai-translator:translate-json --source=en --locale=es --non-interactive --chunk=100
sleep 5

echo ""
echo "Step 5: Translating to French (fr)..."
php artisan ai-translator:translate-json --source=en --locale=fr --non-interactive --chunk=100

echo ""
echo "=== TRANSLATION COMPLETE ==="
echo ""

# Show final counts
echo "Final key counts:"
php -r "
\$en = json_decode(file_get_contents('resources/lang/en.json'), true);
\$nl = json_decode(file_get_contents('resources/lang/nl.json'), true);
\$de = json_decode(file_get_contents('resources/lang/de.json'), true);
\$es = json_decode(file_get_contents('resources/lang/es.json'), true);
\$fr = json_decode(file_get_contents('resources/lang/fr.json'), true);

echo \"  en: \" . count(\$en) . \" keys\n\";
echo \"  nl: \" . count(\$nl) . \" keys\n\";
echo \"  de: \" . count(\$de) . \" keys\n\";
echo \"  es: \" . count(\$es) . \" keys\n\";
echo \"  fr: \" . count(\$fr) . \" keys\n\";
"
```

Make it executable:
```bash
chmod +x translate-new-keys.sh
```

## Translation Configuration

The AI translator is configured in `config/ai-translator.php`:

**Key Settings:**
- **Provider:** Anthropic (Claude)
- **Model:** claude-3-haiku-20240307
- **API Key:** Set in `.env` as `ANTHROPIC_API_KEY`
- **Source Locale:** en (English)
- **Tone:** Friendly, intuitive, informal
- **Addressing Style:** Informal (je/du/tú/tu, not u/Sie/usted/vous)

**Additional Rules Applied:**
```php
'additional_rules' => [
    'default' => [
        "Use a friendly, intuitive, and informal tone of voice. Simple vocabulary is preferred over advanced vocabulary.",
        "Use informal addressing: 'je' in Dutch, 'tu' in French, 'du' in German, 'tú' in Spanish (not formal 'u', 'vous', 'Sie', 'usted').",
    ],
],
```

## Troubleshooting

### Issue: AI Translator Skips All Keys

**Symptom:** Message says "All strings are already translated. Skipping."

**Cause:** The translator considers `key === value` as "already translated"

**Solution:** Remove untranslated keys before running the translator:

```bash
php -r "
\$file = 'resources/lang/nl.json';
\$data = json_decode(file_get_contents(\$file), true);
\$filtered = array_filter(\$data, function(\$value, \$key) {
    return \$value !== \$key; // Remove where key equals value
}, ARRAY_FILTER_USE_BOTH);
file_put_contents(\$file, json_encode(\$filtered, JSON_PRETTY_PRINT | JSON_UNESCAPED_UNICODE | JSON_UNESCAPED_SLASHES) . PHP_EOL);
"
```

### Issue: Translation Files Out of Sync

**Symptom:** Different number of keys in different language files

**Solution:** Run the sync command:
```bash
php artisan ai-translator:sync-json
```

### Issue: API Rate Limiting

**Symptom:** Translation fails with rate limit errors

**Solution:**
1. Reduce chunk size: `--chunk=50`
2. Add delays between translations (see translate-new-keys.sh script)
3. Translate one language at a time with longer delays

### Issue: Some Keys Not Translating

**Symptom:** Some keys remain in English in other language files

**Solution:**
1. Check that the key exists in `en.json`
2. Verify the key format (valid JSON)
3. Check for special characters that might break translation
4. Manually review and re-run translation for specific locale

### Issue: Translations Show Literal Keys Instead of Values

**Symptom:** Translations display literal key names like "messages.login_success" instead of the actual translated text

**Cause:** This occurs when PHP file-based translations (e.g., `resources/lang/en/messages.php`) conflict with JSON translations. The AI translator may have incorrectly added keys from PHP translation files to JSON files with literal key names as values.

**Example of the problem:**
```json
{
  "messages.login_success": "messages.login_success"
}
```

This prevents Laravel from falling back to the PHP translation file.

**Solution:** Remove all conflicting keys that start with `messages.` from all JSON translation files:

```bash
php -r "
\$messagesPhp = require('resources/lang/en/messages.php');
\$languages = ['en', 'nl', 'de', 'es', 'fr'];

// Get list of conflicting keys
\$conflictingKeys = [];
foreach (\$messagesPhp as \$key => \$value) {
    \$conflictingKeys[] = 'messages.' . \$key;
}

echo 'Removing ' . count(\$conflictingKeys) . ' conflicting keys from JSON files...' . PHP_EOL;

foreach (\$languages as \$lang) {
    \$file = 'resources/lang/' . \$lang . '.json';
    \$data = json_decode(file_get_contents(\$file), true);
    \$originalCount = count(\$data);

    foreach (\$conflictingKeys as \$key) {
        if (isset(\$data[\$key])) {
            unset(\$data[\$key]);
        }
    }

    file_put_contents(\$file, json_encode(\$data, JSON_PRETTY_PRINT | JSON_UNESCAPED_UNICODE | JSON_UNESCAPED_SLASHES) . PHP_EOL);

    echo \$lang . '.json: ' . \$originalCount . ' -> ' . count(\$data) . ' keys' . PHP_EOL;
}

echo 'Done! Clear cache with: php artisan config:clear && php artisan cache:clear' . PHP_EOL;
"
```

After removing the conflicting keys, clear Laravel's cache:
```bash
php artisan config:clear && php artisan cache:clear
```

**Important Note:** This project uses BOTH JSON translation files (for simple strings) AND PHP translation files (for dynamic strings with parameters). The `resources/lang/en/messages.php` file (and its counterparts in other languages) should NEVER be duplicated into JSON files. Laravel will automatically use PHP files when the key is not found in JSON files.

## Best Practices

### 1. Always Start with English
- Add all new translation strings to `en.json` first
- Use clear, concise English that translates well
- Avoid idioms or culturally-specific phrases

### 2. Use Consistent Key Naming
```
Good:
  messages.welcome_message
  buttons.submit_form
  errors.validation_failed

Bad:
  welcomeMsg
  btnSubmit
  error_1
```

### 3. Organize Keys Logically
```json
{
  "auth.login": "Log in",
  "auth.logout": "Log out",
  "auth.register": "Register",

  "profile.edit": "Edit profile",
  "profile.delete": "Delete profile",
  "profile.settings": "Profile settings",

  "messages.welcome": "Welcome",
  "messages.goodbye": "Goodbye"
}
```

### 4. Test Before Committing
- Always test translations in the application
- Check all 5 languages
- Verify formatting (capitalization, punctuation)
- Ensure placeholders (`:name`, `:count`) work correctly

### 5. Regular Cleanup
Periodically check for unused translation keys:

```bash
php artisan ai-translator:find-unused --format=table
```

Review and remove unused keys to keep files maintainable.

## Common Translation Patterns

### Simple Strings
```php
__('Welcome to our platform')
```

### With Placeholders
```php
__('Hello, :name!', ['name' => $user->name])
```

### Pluralization
```php
trans_choice('{0} No items|{1} One item|[2,*] :count items', $count)
```

### Conditional Translation
```php
__('messages.' . ($type === 'success' ? 'success_message' : 'error_message'))
```

## Git Workflow

When committing translation updates:

```bash
# Stage all translation files
git add resources/lang/*.json

# Commit with descriptive message
git commit -m "Add translations for new mailing features

- Added 25 new translation keys for mailing management
- Translated to nl, de, es, fr using AI translator
- All files now have 1,414 keys"

# Push changes
git push origin main
```

## Related Files

- **Config:** `config/ai-translator.php`
- **Translation Files:** `resources/lang/*.json`
- **Helper Scripts:**
  - `retranslate-informal.sh` - Re-translate all languages with informal style
  - `translate-new-keys.sh` - Translate only new keys (create this)
  - `sync-translation-files.php` - Manual sync script (backup method)

## Quick Command Reference

```bash
# Sync all translation files
php artisan ai-translator:sync-json

# Translate to specific language
php artisan ai-translator:translate-json --source=en --locale=nl --non-interactive --chunk=100

# Find unused translation keys
php artisan ai-translator:find-unused

# Count keys in all files
php -r "\$en = json_decode(file_get_contents('resources/lang/en.json'), true); echo count(\$en);"

# Clear Laravel cache
php artisan config:clear && php artisan cache:clear
```

## Environment Requirements

- **PHP:** 8.1+
- **Laravel:** 9+
- **Package:** kargnas/laravel-ai-translator
- **API Key:** Anthropic Claude API (set in `.env` as `ANTHROPIC_API_KEY`)
- **Internet:** Required for AI translation API calls

## Support

For issues with:
- **Laravel AI Translator Package:** https://github.com/kargnas/laravel-ai-translator
- **Translation Process:** Review this guide or check logs in `/tmp/`
- **API Issues:** Check Anthropic API status and your API key validity