No OneTemporary
Actions

Authored By

Unknown

Size

60 KB

Referenced Files

None

Subscribers

None

View Options

	diff --git a/README.md b/README.md
	index 6e74f3a..2507db7 100644
	--- a/README.md
	+++ b/README.md
	@@ -1,140 +1,185 @@
	# mattata

	A feature-rich Telegram group management and utility bot, written in Lua.

	## Features

	- Group Administration - Ban, kick, mute, warn, tempban, tempmute, promote, demote, trust
	- Federation System - Cross-group ban management with federated admin networks
	- Captcha Verification - Challenge new members with built-in or custom captcha
	- Anti-Spam - Rate limiting, word filters, link filtering, auto-delete
	- 100+ Plugins - Weather, translate, search, currency, Wikipedia, AI chat, and more
	- Async HTTP - Non-blocking HTTP client for plugin API calls via copas
	- Stored Procedures - All database operations use PostgreSQL stored procedures
	- Forum Topics - Full support for forum topic management and slow mode
	- Reaction Karma - Track karma via message reactions
	- RSS Feeds - Subscribe to and monitor RSS/Atom feeds
	- Scheduled Messages - Queue messages for future delivery
	- Inline Queries - Inline search and sharing across chats
	- QR Codes - Generate QR codes from text
	- Multi-Language - 10 language packs included (EN, DE, AR, PL, PT, TR, Scottish)
	- PostgreSQL + Redis - PostgreSQL for persistent data, Redis for caching
	- Hot-Reloadable Plugins - Reload plugins without restarting the bot
	- Docker Ready - One command deployment with Docker Compose

	## Quick Start (Docker)

	```bash
	cp .env.example .env
	# Edit .env and set BOT_TOKEN
	docker compose up -d
	```

	## Quick Start (Manual)

	### Prerequisites
	- Lua 5.3+
	- LuaRocks
	- PostgreSQL 14+
	- Redis 7+

	### Installation

	```bash
	# Install Lua dependencies
	luarocks install telegram-bot-lua
	luarocks install pgmoon
	luarocks install redis-lua
	luarocks install dkjson
	luarocks install luautf8

	# Configure
	cp .env.example .env
	# Edit .env with your settings

	# Start
	lua main.lua
	```

	+## Upgrading from v1.5
	+
	+If you are upgrading from mattata v1.5, the migration is automatic. On first boot, mattata v2 detects v1.5 data and imports it into the new schema.
	+
	+### What gets migrated
	+
	+- Chat settings (antilink, welcome, captcha, max warnings, etc.)
	+- Welcome messages
	+- Rules
	+- Warnings (per-user warning counts)
	+- Disabled plugins (with name mapping to v2 plugin names)
	+- Filters (word filters with actions)
	+- Triggers (auto-responses)
	+- Bans (group blocklist entries)
	+- Configuration (`configuration.lua` converted to `.env` format)
	+
	+### How it works
	+
	+1. On startup, mattata scans Redis for v1-era key patterns and checks for `configuration.lua`
	+2. If v1 data is found, it imports everything into PostgreSQL via a single transaction
	+3. The migration is recorded in `schema_migrations` so it only runs once
	+4. After success, v1 Redis keys are cleaned up and `configuration.lua` is renamed to `configuration.lua.v1.bak`
	+
	+### Prerequisites
	+
	+- Your v1.5 Redis instance must still be accessible (same host/port/db)
	+- PostgreSQL must be running and configured in `.env` (or the migration will create `.env` from `configuration.lua`)
	+
	+### Config mapping reference
	+
	+\| v1 (`configuration.lua`) \| v2 (`.env`) \|
	+\|---------------------------\|-------------\|
	+\| `bot_token` \| `BOT_TOKEN` \|
	+\| `admins` \| `BOT_ADMINS` \|
	+\| `redis.host` \| `REDIS_HOST` \|
	+\| `redis.port` \| `REDIS_PORT` \|
	+\| `redis.password` \| `REDIS_PASSWORD` \|
	+\| `keys.lastfm` \| `LASTFM_API_KEY` \|
	+\| `keys.youtube` \| `YOUTUBE_API_KEY` \|
	+\| `keys.weather` \| `OPENWEATHERMAP_API_KEY` \|
	+\| `keys.spotify_client_id` \| `SPOTIFY_CLIENT_ID` \|
	+\| `keys.spotify_client_secret` \| `SPOTIFY_CLIENT_SECRET` \|
	+\| `keys.spamwatch` \| `SPAMWATCH_TOKEN` \|
	+\| `log_channel` / `log_chat` \| `LOG_CHAT` \|
	+
	## Configuration

	All configuration is managed through environment variables. See `.env.example` for the full reference.

	### Required
	\| Variable \| Description \|
	\|----------\|-------------\|
	\| `BOT_TOKEN` \| Telegram Bot API token from @BotFather \|

	### Optional API Keys
	\| Variable \| Description \| Used By \|
	\|----------\|-------------\|---------\|
	\| `LASTFM_API_KEY` \| Last.fm API key \| `/lastfm`, `/np` \|
	\| `YOUTUBE_API_KEY` \| YouTube Data API v3 key \| `/youtube` \|
	\| `SPOTIFY_CLIENT_ID` \| Spotify app client ID \| `/spotify` \|
	\| `SPOTIFY_CLIENT_SECRET` \| Spotify app client secret \| `/spotify` \|
	\| `SPAMWATCH_TOKEN` \| SpamWatch API token \| Anti-spam \|
	\| `OPENAI_API_KEY` \| OpenAI API key \| `/ai` \|
	\| `ANTHROPIC_API_KEY` \| Anthropic API key \| `/ai` \|

	## Architecture

	```
	mattata/
	├── main.lua # Entry point
	├── src/
	│ ├── core/ # Framework modules
	│ │ ├── config.lua # .env configuration reader
	│ │ ├── loader.lua # Plugin discovery & hot-reload
	│ │ ├── router.lua # Event dispatch
	│ │ ├── middleware.lua # Middleware pipeline
	│ │ ├── database.lua # PostgreSQL (pgmoon)
	│ │ ├── redis.lua # Redis connection
	│ │ ├── http.lua # Async HTTP client (copas)
	│ │ ├── permissions.lua # Admin/mod/trusted checks
	│ │ ├── session.lua # Redis session/cache management
	│ │ ├── i18n.lua # Language manager
	│ │ └── logger.lua # Structured logging
	│ ├── middleware/ # Middleware chain
	│ ├── plugins/ # Plugin categories
	│ │ ├── admin/ # Group management (35+ plugins)
	│ │ ├── utility/ # Tools & info (33+ plugins)
	│ │ ├── fun/ # Entertainment (16 plugins)
	│ │ ├── media/ # Media search (7 plugins)
	│ │ └── ai/ # LLM integration
	│ ├── db/migrations/ # PostgreSQL schema migrations
	│ ├── languages/ # 10 language packs
	│ └── data/ # Static data (slaps, join messages)
	├── docker-compose.yml
	├── docker-compose.matticate.yml
	├── Dockerfile
	└── .env.example
	```

	## Plugin Development

	Plugins follow a simple contract:

	```lua
	local plugin = {}
	plugin.name = 'myplugin'
	plugin.category = 'utility'
	plugin.description = 'Does something useful'
	plugin.commands = { 'mycommand', 'alias' }
	plugin.help = '/mycommand <args> - Does the thing.'

	function plugin.on_message(api, message, ctx)
	return api.send_message(message.chat.id, 'Hello!')
	end

	return plugin
	```

	Add your plugin to the category's `init.lua` manifest and it will be auto-loaded.

	## License

	MIT License - see [LICENSE](LICENSE) for details.

	## Credits

	Created by [Matt Hesketh](https://github.com/wrxck).
	diff --git a/main.lua b/main.lua
	index 7cde9aa..35bf9a8 100644
	--- a/main.lua
	+++ b/main.lua
	@@ -1,120 +1,138 @@
	--[[
	_ _ _
	_ __ ___ __ _\| \|_\| \|_ __ _\| \|_ __ _
	\| '_ ` _ \ / _` \| __\| __/ _` \| __/ _` \|
	\| \| \| \| \| \| (_\| \| \|_\| \|\| (_\| \| \|\| (_\| \|
	\|_\| \|_\| \|_\|\__,_\|\__\|\__\__,_\|\__\__,_\|

	v2.1

	Copyright 2020-2026 Matthew Hesketh <matthew@matthewhesketh.com>
	See LICENSE for details
	]]

	local config = require('src.core.config')
	local logger = require('src.core.logger')
	local database = require('src.core.database')
	local redis = require('src.core.redis')
	local session = require('src.core.session')
	local i18n = require('src.core.i18n')
	local loader = require('src.core.loader')
	local router = require('src.core.router')
	local migrations = require('src.db.init')

	-- 1. Load configuration
	config.load('.env')
	logger.init()
	logger.info('mattata v%s starting...', config.VERSION)

	-- 2. Validate required config
	assert(config.bot_token(), 'BOT_TOKEN is required. Set it in .env or as an environment variable.')

	-- 3. Configure telegram-bot-lua
	local api = require('telegram-bot-lua').configure(config.bot_token())
	local tools = require('telegram-bot-lua.tools')
	logger.info('Bot: @%s (%s) [%d]', api.info.username, api.info.first_name, api.info.id)

	-- 4. Connect to PostgreSQL
	local db_ok, db_err = database.connect()
	if not db_ok then
	logger.error('Cannot start without PostgreSQL: %s', tostring(db_err))
	os.exit(1)
	end

	-- 5. Run database migrations
	migrations.run(database)

	-- 6. Connect to Redis
	local redis_ok, redis_err = redis.connect()
	if not redis_ok then
	logger.error('Cannot start without Redis: %s', tostring(redis_err))
	os.exit(1)
	end
	session.init(redis)

	+-- 6b. Check for v1.5 data migration
	+local migrate = require('src.core.migrate')
	+local v1_check = migrate.check(redis)
	+if v1_check.detected then
	+ logger.info('v1.5 installation detected (%d Redis keys). Running migration...', v1_check.key_count)
	+ local v1_result = migrate.run(database, redis)
	+ if v1_result.success then
	+ if v1_result.already_migrated then
	+ logger.debug('v1.5 migration already applied, skipping')
	+ else
	+ logger.info('v1.5 migration complete: %d records imported, %d keys cleaned',
	+ v1_result.records_imported, v1_result.keys_cleaned)
	+ end
	+ else
	+ logger.warn('v1.5 migration errors: %s', table.concat(v1_result.errors, '; '))
	+ end
	+end
	+
	-- 7. Load languages
	i18n.init()

	-- 8. Load all plugins
	loader.init(api, database, redis)

	-- 9. Build context factory and start router
	local ctx_base = {
	api = api,
	tools = tools,
	db = database,
	redis = redis,
	session = session,
	config = config,
	i18n = i18n,
	permissions = require('src.core.permissions'),
	logger = logger
	}

	router.init(api, tools, loader, ctx_base)

	-- 10. Register bot command menu with Telegram
	local json = require('dkjson')
	local user_commands = {}
	local admin_commands = {}
	for _, plugin in ipairs(loader.get_plugins()) do
	if plugin.commands and #plugin.commands > 0 and plugin.description and plugin.description ~= '' then
	local entry = { command = plugin.commands[1], description = plugin.description }
	if plugin.admin_only or plugin.global_admin_only then
	table.insert(admin_commands, entry)
	else
	table.insert(user_commands, entry)
	end
	end
	end
	-- Set default commands (visible to all users)
	api.set_my_commands(json.encode(user_commands))
	-- Set admin commands (visible to group admins only)
	if #admin_commands > 0 then
	-- Merge user + admin so admins see everything
	local all_commands = {}
	for _, cmd in ipairs(user_commands) do table.insert(all_commands, cmd) end
	for _, cmd in ipairs(admin_commands) do table.insert(all_commands, cmd) end
	api.set_my_commands(json.encode(all_commands), { type = 'all_chat_administrators' })
	end
	logger.info('Registered %d user commands and %d admin commands with Telegram', #user_commands, #admin_commands)

	-- 11. Notify admins
	local info_msg = string.format(
	'<pre>mattata v%s connected!\n\n Username: @%s\n Name: %s\n ID: %d\n Plugins: %d</pre>',
	config.VERSION,
	tools.escape_html(api.info.username),
	tools.escape_html(api.info.first_name),
	api.info.id,
	loader.count()
	)
	if config.log_chat() then
	api.send_message(config.log_chat(), info_msg, 'html')
	end
	for _, admin_id in ipairs(config.bot_admins()) do
	api.send_message(admin_id, info_msg, 'html')
	end

	-- 12. Start the bot
	logger.info('Starting main loop...')
	router.run()
	diff --git a/spec/core/migrate_spec.lua b/spec/core/migrate_spec.lua
	new file mode 100644
	index 0000000..a86d46f
	--- /dev/null
	+++ b/spec/core/migrate_spec.lua
	@@ -0,0 +1,547 @@
	+--[[
	+ Tests for src/core/migrate.lua - v1.5 to v2.x data migration
	+]]
	+
	+describe('core.migrate', function()
	+ local migrate
	+ local mock_db = require('spec.helpers.mock_db')
	+ local mock_redis = require('spec.helpers.mock_redis')
	+ local db, redis
	+ local tmpdir = os.tmpname():match('(.*/)')
	+
	+ -- Helper: write a temp file and return its path
	+ local function write_temp(filename, content)
	+ local path = tmpdir .. filename
	+ local f = io.open(path, 'w')
	+ f:write(content)
	+ f:close()
	+ return path
	+ end
	+
	+ -- Helper: check if a file exists
	+ local function file_exists(path)
	+ local f = io.open(path, 'r')
	+ if f then f:close(); return true end
	+ return false
	+ end
	+
	+ -- Helper: populate v1 Redis keys for scan detection
	+ -- mock_redis.scan only checks redis.store, so we must set placeholder values there.
	+ -- The actual data is in redis.hashes / redis.sets / redis.store (for strings).
	+ local function set_v1_hash(r, key, data)
	+ r.store[key] = 'hash' -- placeholder for scan detection
	+ r.hashes[key] = data
	+ end
	+
	+ local function set_v1_string(r, key, value)
	+ r.store[key] = value
	+ end
	+
	+ local function set_v1_set(r, key, members)
	+ r.store[key] = 'set' -- placeholder for scan detection
	+ r.sets[key] = {}
	+ for _, m in ipairs(members) do
	+ r.sets[key][tostring(m)] = true
	+ end
	+ end
	+
	+ before_each(function()
	+ package.loaded['src.core.migrate'] = nil
	+ package.loaded['src.core.logger'] = {
	+ info = function() end,
	+ warn = function() end,
	+ error = function() end,
	+ debug = function() end,
	+ }
	+ migrate = require('src.core.migrate')
	+ db = mock_db.new()
	+ redis = mock_redis.new()
	+ end)
	+
	+ after_each(function()
	+ db.reset()
	+ redis.reset()
	+ end)
	+
	+ -- ================================================================
	+ -- Detection tests
	+ -- ================================================================
	+ describe('check()', function()
	+ it('returns detected=false on clean install', function()
	+ local result = migrate.check(redis, { v1_config_path = '/nonexistent/config.lua' })
	+ assert.is_false(result.detected)
	+ assert.is_nil(result.config_file)
	+ assert.are.equal(0, result.key_count)
	+ end)
	+
	+ it('detects v1 config file', function()
	+ local path = write_temp('test_v1_config.lua', 'return { bot_token = "123:ABC", admins = {111} }')
	+ local result = migrate.check(redis, { v1_config_path = path })
	+ assert.is_true(result.detected)
	+ assert.are.equal(path, result.config_file)
	+ os.remove(path)
	+ end)
	+
	+ it('detects v1 Redis keys', function()
	+ set_v1_hash(redis, 'chat:-100123:settings', { welcome = 'true' })
	+ set_v1_string(redis, 'chat:-100123:welcome', 'Hello!')
	+ local result = migrate.check(redis, { v1_config_path = '/nonexistent.lua' })
	+ assert.is_true(result.detected)
	+ assert.are.equal(2, result.key_count)
	+ end)
	+
	+ it('counts keys accurately across categories', function()
	+ set_v1_hash(redis, 'chat:-100123:settings', {})
	+ set_v1_string(redis, 'chat:-100123:welcome', 'Hi')
	+ set_v1_string(redis, 'chat:-100123:rules', 'Rule 1')
	+ set_v1_string(redis, 'chat:-100123:warnings:456', '3')
	+ set_v1_hash(redis, 'chat:-100123:filters', {})
	+ local result = migrate.check(redis, { v1_config_path = '/nonexistent.lua' })
	+ assert.are.equal(5, result.key_count)
	+ end)
	+
	+ it('does not false-positive on v2 Redis key patterns', function()
	+ -- v2 patterns: cache:setting:, disabled_plugins:
	+ redis.store['cache:setting:-100123:welcome_enabled'] = 'true'
	+ redis.store['disabled_plugins:-100123'] = 'set'
	+ local result = migrate.check(redis, { v1_config_path = '/nonexistent.lua' })
	+ assert.is_false(result.detected)
	+ assert.are.equal(0, result.key_count)
	+ end)
	+ end)
	+
	+ -- ================================================================
	+ -- Config detection tests
	+ -- ================================================================
	+ describe('detect_v1_config()', function()
	+ it('parses valid configuration.lua', function()
	+ local path = write_temp('test_detect_valid.lua',
	+ 'return { bot_token = "123:ABC", admins = {111, 222}, redis = { host = "localhost", port = 6379 } }')
	+ local config = migrate.detect_v1_config(path)
	+ assert.is_not_nil(config)
	+ assert.are.equal('123:ABC', config.bot_token)
	+ assert.are.equal(2, #config.admins)
	+ os.remove(path)
	+ end)
	+
	+ it('returns nil for missing file', function()
	+ local config = migrate.detect_v1_config('/nonexistent/config.lua')
	+ assert.is_nil(config)
	+ end)
	+
	+ it('returns nil for malformed Lua', function()
	+ local path = write_temp('test_detect_malformed.lua', 'this is not valid lua {{{')
	+ local config = migrate.detect_v1_config(path)
	+ assert.is_nil(config)
	+ os.remove(path)
	+ end)
	+
	+ it('handles config with nested keys table', function()
	+ local path = write_temp('test_detect_keys.lua',
	+ 'return { bot_token = "123:ABC", admins = {111}, keys = { lastfm = "abc123", youtube = "xyz789" } }')
	+ local config = migrate.detect_v1_config(path)
	+ assert.is_not_nil(config)
	+ assert.are.equal('abc123', config.keys.lastfm)
	+ assert.are.equal('xyz789', config.keys.youtube)
	+ os.remove(path)
	+ end)
	+ end)
	+
	+ -- ================================================================
	+ -- Config conversion tests
	+ -- ================================================================
	+ describe('convert_config()', function()
	+ it('converts bot_token and admins', function()
	+ local env = migrate.convert_config({
	+ bot_token = '123:ABC',
	+ admins = { 111, 222 },
	+ })
	+ assert.matches('BOT_TOKEN=123:ABC', env)
	+ assert.matches('BOT_ADMINS=111,222', env)
	+ end)
	+
	+ it('converts Redis config', function()
	+ local env = migrate.convert_config({
	+ bot_token = '123:ABC',
	+ admins = { 111 },
	+ redis = { host = '10.0.0.1', port = 6380, password = 'secret' },
	+ })
	+ assert.matches('REDIS_HOST=10.0.0.1', env)
	+ assert.matches('REDIS_PORT=6380', env)
	+ assert.matches('REDIS_PASSWORD=secret', env)
	+ end)
	+
	+ it('converts API keys', function()
	+ local env = migrate.convert_config({
	+ bot_token = '123:ABC',
	+ admins = { 111 },
	+ keys = {
	+ lastfm = 'lf_key',
	+ youtube = 'yt_key',
	+ weather = 'wx_key',
	+ spotify_client_id = 'sp_id',
	+ spotify_client_secret = 'sp_secret',
	+ spamwatch = 'sw_token',
	+ },
	+ })
	+ assert.matches('LASTFM_API_KEY=lf_key', env)
	+ assert.matches('YOUTUBE_API_KEY=yt_key', env)
	+ assert.matches('OPENWEATHERMAP_API_KEY=wx_key', env)
	+ assert.matches('SPOTIFY_CLIENT_ID=sp_id', env)
	+ assert.matches('SPOTIFY_CLIENT_SECRET=sp_secret', env)
	+ assert.matches('SPAMWATCH_TOKEN=sw_token', env)
	+ end)
	+
	+ it('handles missing optional keys gracefully', function()
	+ local env = migrate.convert_config({
	+ bot_token = '123:ABC',
	+ admins = { 111 },
	+ })
	+ assert.matches('BOT_TOKEN=123:ABC', env)
	+ assert.is_not.matches('REDIS_HOST', env)
	+ assert.is_not.matches('LASTFM_API_KEY', env)
	+ end)
	+
	+ it('produces valid key=value format', function()
	+ local env = migrate.convert_config({
	+ bot_token = '123:ABC',
	+ admins = { 111 },
	+ log_chat = '-100999',
	+ })
	+ -- Every non-comment non-empty line should be KEY=VALUE
	+ for line in env:gmatch('[^\n]+') do
	+ if not line:match('^#') and line ~= '' then
	+ assert.matches('^[A-Z_]+=.+$', line)
	+ end
	+ end
	+ end)
	+
	+ it('skips empty/nil values', function()
	+ local env = migrate.convert_config({
	+ bot_token = '123:ABC',
	+ admins = { 111 },
	+ redis = { host = '', port = 6379 },
	+ })
	+ assert.is_not.matches('REDIS_HOST=\n', env)
	+ end)
	+ end)
	+
	+ -- ================================================================
	+ -- Import tests: chat settings
	+ -- ================================================================
	+ describe('import_chat_settings()', function()
	+ it('imports hash fields and maps setting names', function()
	+ set_v1_hash(redis, 'chat:-100123:settings', {
	+ welcome = 'true',
	+ antilink = 'true',
	+ ['max warnings'] = '5',
	+ })
	+ local keys = { 'chat:-100123:settings' }
	+ local count = migrate.import_chat_settings(db, redis, keys)
	+ assert.are.equal(3, count)
	+ -- Check that sp_upsert_chat was called (ensure_chat)
	+ assert.is_true(db.has_query('sp_upsert_chat'))
	+ -- Check that sp_upsert_chat_setting_if_missing was called
	+ assert.is_true(db.has_query('sp_upsert_chat_setting_if_missing'))
	+ end)
	+
	+ it('maps v1 names to v2 names', function()
	+ set_v1_hash(redis, 'chat:-100123:settings', { antilink = 'true' })
	+ migrate.import_chat_settings(db, redis, { 'chat:-100123:settings' })
	+ -- Find the call with the mapped name
	+ local found = false
	+ for _, q in ipairs(db.queries) do
	+ if q.func_name == 'sp_upsert_chat_setting_if_missing' and q.params then
	+ if q.params[2] == 'antilink_enabled' then
	+ found = true
	+ break
	+ end
	+ end
	+ end
	+ assert.is_true(found, 'Expected antilink to be mapped to antilink_enabled')
	+ end)
	+ end)
	+
	+ -- ================================================================
	+ -- Import tests: welcome messages
	+ -- ================================================================
	+ describe('import_welcome_messages()', function()
	+ it('imports string to welcome_messages table', function()
	+ set_v1_string(redis, 'chat:-100123:welcome', 'Welcome to the group!')
	+ local count = migrate.import_welcome_messages(db, redis, { 'chat:-100123:welcome' })
	+ assert.are.equal(1, count)
	+ assert.is_true(db.has_query('sp_upsert_welcome_message'))
	+ end)
	+
	+ it('skips empty welcome messages', function()
	+ set_v1_string(redis, 'chat:-100123:welcome', '')
	+ local count = migrate.import_welcome_messages(db, redis, { 'chat:-100123:welcome' })
	+ assert.are.equal(0, count)
	+ end)
	+ end)
	+
	+ -- ================================================================
	+ -- Import tests: rules
	+ -- ================================================================
	+ describe('import_rules()', function()
	+ it('imports string to rules table', function()
	+ set_v1_string(redis, 'chat:-100123:rules', '1. Be nice\n2. No spam')
	+ local count = migrate.import_rules(db, redis, { 'chat:-100123:rules' })
	+ assert.are.equal(1, count)
	+ assert.is_true(db.has_query('sp_upsert_rules'))
	+ end)
	+
	+ it('skips empty rules', function()
	+ set_v1_string(redis, 'chat:-100123:rules', '')
	+ local count = migrate.import_rules(db, redis, { 'chat:-100123:rules' })
	+ assert.are.equal(0, count)
	+ end)
	+ end)
	+
	+ -- ================================================================
	+ -- Import tests: warnings
	+ -- ================================================================
	+ describe('import_warnings()', function()
	+ it('sets v2 Redis hash and inserts DB rows', function()
	+ set_v1_string(redis, 'chat:-100123:warnings:456', '3')
	+ local count = migrate.import_warnings(db, redis, { 'chat:-100123:warnings:456' })
	+ assert.are.equal(3, count)
	+ -- Check v2 Redis hash was set
	+ assert.are.equal('3', redis.hashes['chat:-100123:456'] and redis.hashes['chat:-100123:456']['warnings'])
	+ -- Check DB calls (sp_insert_warning called 3 times)
	+ local warning_calls = 0
	+ for _, q in ipairs(db.queries) do
	+ if q.func_name == 'sp_insert_warning' then
	+ warning_calls = warning_calls + 1
	+ end
	+ end
	+ assert.are.equal(3, warning_calls)
	+ end)
	+
	+ it('skips zero warnings', function()
	+ set_v1_string(redis, 'chat:-100123:warnings:456', '0')
	+ local count = migrate.import_warnings(db, redis, { 'chat:-100123:warnings:456' })
	+ assert.are.equal(0, count)
	+ end)
	+ end)
	+
	+ -- ================================================================
	+ -- Import tests: disabled plugins
	+ -- ================================================================
	+ describe('import_disabled_plugins()', function()
	+ it('inserts to DB table and Redis set', function()
	+ set_v1_hash(redis, 'chat:-100123:disabled_plugins', {
	+ welcome = 'true',
	+ lastfm = 'true',
	+ })
	+ local count = migrate.import_disabled_plugins(db, redis, { 'chat:-100123:disabled_plugins' })
	+ assert.are.equal(2, count)
	+ -- Check Redis set was populated with v2 mapped names
	+ assert.are.equal(1, redis.sismember('disabled_plugins:-100123', 'greeting'))
	+ assert.are.equal(1, redis.sismember('disabled_plugins:-100123', 'lastfm'))
	+ end)
	+
	+ it('maps v1 plugin names to v2 names', function()
	+ set_v1_hash(redis, 'chat:-100123:disabled_plugins', { administration = 'true' })
	+ migrate.import_disabled_plugins(db, redis, { 'chat:-100123:disabled_plugins' })
	+ -- "administration" should map to "admin"
	+ assert.are.equal(1, redis.sismember('disabled_plugins:-100123', 'admin'))
	+ end)
	+ end)
	+
	+ -- ================================================================
	+ -- Import tests: filters
	+ -- ================================================================
	+ describe('import_filters()', function()
	+ it('imports pattern/action pairs', function()
	+ set_v1_hash(redis, 'chat:-100123:filters', {
	+ ['bad word'] = 'delete',
	+ ['spam link'] = 'ban',
	+ })
	+ local count = migrate.import_filters(db, redis, { 'chat:-100123:filters' })
	+ assert.are.equal(2, count)
	+ assert.is_true(db.has_query('sp_insert_filter'))
	+ end)
	+
	+ it('handles empty filter hash', function()
	+ set_v1_hash(redis, 'chat:-100123:filters', {})
	+ local count = migrate.import_filters(db, redis, { 'chat:-100123:filters' })
	+ assert.are.equal(0, count)
	+ end)
	+ end)
	+
	+ -- ================================================================
	+ -- Import tests: triggers
	+ -- ================================================================
	+ describe('import_triggers()', function()
	+ it('imports pattern/response pairs', function()
	+ set_v1_hash(redis, 'chat:-100123:triggers', {
	+ ['hello'] = 'Hi there!',
	+ ['bye'] = 'Goodbye!',
	+ })
	+ local count = migrate.import_triggers(db, redis, { 'chat:-100123:triggers' })
	+ assert.are.equal(2, count)
	+ assert.is_true(db.has_query('sp_insert_trigger'))
	+ end)
	+
	+ it('handles empty trigger hash', function()
	+ set_v1_hash(redis, 'chat:-100123:triggers', {})
	+ local count = migrate.import_triggers(db, redis, { 'chat:-100123:triggers' })
	+ assert.are.equal(0, count)
	+ end)
	+ end)
	+
	+ -- ================================================================
	+ -- Import tests: bans
	+ -- ================================================================
	+ describe('import_bans()', function()
	+ it('imports set members to blocklist', function()
	+ set_v1_set(redis, 'chat:-100123:bans', { 111, 222, 333 })
	+ local count = migrate.import_bans(db, redis, { 'chat:-100123:bans' })
	+ assert.are.equal(3, count)
	+ assert.is_true(db.has_query('sp_upsert_blocklist_entry'))
	+ end)
	+
	+ it('handles empty ban set', function()
	+ set_v1_set(redis, 'chat:-100123:bans', {})
	+ local count = migrate.import_bans(db, redis, { 'chat:-100123:bans' })
	+ assert.are.equal(0, count)
	+ end)
	+ end)
	+
	+ -- ================================================================
	+ -- Pipeline tests
	+ -- ================================================================
	+ describe('run()', function()
	+ it('succeeds end-to-end with v1 data', function()
	+ set_v1_hash(redis, 'chat:-100123:settings', { welcome = 'true' })
	+ set_v1_string(redis, 'chat:-100123:welcome', 'Hello!')
	+ set_v1_string(redis, 'chat:-100123:rules', 'Be nice')
	+ set_v1_hash(redis, 'chat:-100123:filters', { spam = 'delete' })
	+ set_v1_hash(redis, 'chat:-100123:triggers', { hi = 'hello' })
	+ set_v1_set(redis, 'chat:-100123:bans', { 999 })
	+
	+ local result = migrate.run(db, redis, { v1_config_path = '/nonexistent.lua' })
	+ assert.is_true(result.success)
	+ assert.is_false(result.already_migrated)
	+ assert.is_true(result.records_imported > 0)
	+ end)
	+
	+ it('is idempotent via schema_migrations check', function()
	+ -- Simulate already-migrated state
	+ db.set_next_result({ { ['?column?'] = 1 } })
	+ local result = migrate.run(db, redis, { v1_config_path = '/nonexistent.lua' })
	+ assert.is_true(result.success)
	+ assert.is_true(result.already_migrated)
	+ end)
	+
	+ it('dry_run mode does not write to DB', function()
	+ set_v1_hash(redis, 'chat:-100123:settings', { welcome = 'true' })
	+ local result = migrate.run(db, redis, {
	+ dry_run = true,
	+ v1_config_path = '/nonexistent.lua',
	+ })
	+ assert.is_true(result.success)
	+ -- Should not have any BEGIN/COMMIT queries
	+ assert.is_false(db.has_query('BEGIN'))
	+ end)
	+
	+ it('skip_cleanup preserves v1 keys', function()
	+ set_v1_hash(redis, 'chat:-100123:settings', { welcome = 'true' })
	+ local result = migrate.run(db, redis, {
	+ skip_cleanup = true,
	+ v1_config_path = '/nonexistent.lua',
	+ })
	+ assert.is_true(result.success)
	+ assert.are.equal(0, result.keys_cleaned)
	+ -- Key should still exist
	+ assert.is_not_nil(redis.store['chat:-100123:settings'])
	+ end)
	+
	+ it('cleans up v1 keys after success', function()
	+ set_v1_hash(redis, 'chat:-100123:settings', { welcome = 'true' })
	+ set_v1_string(redis, 'chat:-100123:welcome', 'Hello!')
	+ local result = migrate.run(db, redis, { v1_config_path = '/nonexistent.lua' })
	+ assert.is_true(result.success)
	+ assert.are.equal(2, result.keys_cleaned)
	+ -- Keys should be deleted
	+ assert.is_nil(redis.store['chat:-100123:settings'])
	+ assert.is_nil(redis.store['chat:-100123:welcome'])
	+ end)
	+
	+ it('rolls back on DB error', function()
	+ set_v1_hash(redis, 'chat:-100123:settings', { welcome = 'true' })
	+ -- Make db.call throw an error
	+ local orig_call = db.call
	+ db.call = function(func_name, params)
	+ if func_name == 'sp_upsert_chat' then
	+ error('simulated DB error')
	+ end
	+ return orig_call(func_name, params)
	+ end
	+ local result = migrate.run(db, redis, { v1_config_path = '/nonexistent.lua' })
	+ assert.is_false(result.success)
	+ assert.is_true(#result.errors > 0)
	+ assert.matches('simulated DB error', result.errors[1])
	+ -- ROLLBACK should have been issued
	+ assert.is_true(db.has_query('ROLLBACK'))
	+ end)
	+
	+ it('reports errors without crashing', function()
	+ set_v1_hash(redis, 'chat:-100123:settings', { welcome = 'true' })
	+ local orig_call = db.call
	+ db.call = function(func_name, params)
	+ if func_name == 'sp_upsert_chat_setting_if_missing' then
	+ error('constraint violation')
	+ end
	+ return orig_call(func_name, params)
	+ end
	+ local result = migrate.run(db, redis, { v1_config_path = '/nonexistent.lua' })
	+ assert.is_false(result.success)
	+ assert.is_true(#result.errors > 0)
	+ end)
	+
	+ it('is a no-op when no v1 data detected', function()
	+ local result = migrate.run(db, redis, { v1_config_path = '/nonexistent.lua' })
	+ assert.is_true(result.success)
	+ assert.are.equal(0, result.records_imported)
	+ assert.are.equal(0, result.keys_cleaned)
	+ end)
	+ end)
	+
	+ -- ================================================================
	+ -- Cleanup tests
	+ -- ================================================================
	+ describe('cleanup_v1_keys()', function()
	+ it('deletes all categorized keys', function()
	+ local keys = {
	+ settings = { 'chat:-100123:settings', 'chat:-100456:settings' },
	+ welcome = { 'chat:-100123:welcome' },
	+ rules = {},
	+ _total = 3,
	+ }
	+ -- Populate the store so del has something to remove
	+ redis.store['chat:-100123:settings'] = 'hash'
	+ redis.store['chat:-100456:settings'] = 'hash'
	+ redis.store['chat:-100123:welcome'] = 'Hello'
	+
	+ local count = migrate.cleanup_v1_keys(redis, keys)
	+ assert.are.equal(3, count)
	+ assert.is_nil(redis.store['chat:-100123:settings'])
	+ assert.is_nil(redis.store['chat:-100456:settings'])
	+ assert.is_nil(redis.store['chat:-100123:welcome'])
	+ end)
	+
	+ it('returns accurate count', function()
	+ local keys = {
	+ settings = { 'chat:-100123:settings' },
	+ welcome = { 'chat:-100123:welcome' },
	+ rules = { 'chat:-100123:rules' },
	+ warnings = { 'chat:-100123:warnings:456' },
	+ _total = 4,
	+ }
	+ local count = migrate.cleanup_v1_keys(redis, keys)
	+ assert.are.equal(4, count)
	+ end)
	+ end)
	+end)
	diff --git a/src/core/migrate.lua b/src/core/migrate.lua
	new file mode 100644
	index 0000000..8f10921
	--- /dev/null
	+++ b/src/core/migrate.lua
	@@ -0,0 +1,571 @@
	+--[[
	+ mattata v2.1 - v1.5 Data Migration
	+ Detects v1.5 installations and imports data into the v2 schema.
	+ Handles: config conversion, chat settings, welcome messages, rules,
	+ warnings, disabled plugins, filters, triggers, and bans.
	+]]
	+
	+local migrate = {}
	+
	+local logger = require('src.core.logger')
	+
	+-- v1 setting names -> v2 setting names
	+local SETTING_KEY_MAP = {
	+ ['antilink'] = 'antilink_enabled',
	+ ['welcome'] = 'welcome_enabled',
	+ ['captcha'] = 'captcha_enabled',
	+ ['antibot'] = 'antibot_enabled',
	+ ['antiarabic'] = 'antiarabic_enabled',
	+ ['antiflood'] = 'antiflood_enabled',
	+ ['antispam'] = 'antispam_enabled',
	+ ['antiforward'] = 'antiforward_enabled',
	+ ['rtl'] = 'rtl_enabled',
	+ ['max warnings'] = 'max_warnings',
	+ ['delete commands'] = 'delete_commands',
	+ ['force group language'] = 'force_group_language',
	+ ['language'] = 'language',
	+ ['log admin actions'] = 'log_admin_actions',
	+ ['welcome message'] = 'welcome_enabled',
	+ ['use administration'] = 'use_administration',
	+}
	+
	+-- v1 plugin names -> v2 plugin names (most are identical)
	+local PLUGIN_NAME_MAP = {
	+ ['administration'] = 'admin',
	+ ['antispam'] = 'antispam',
	+ ['captcha'] = 'captcha',
	+ ['welcome'] = 'greeting',
	+ ['lastfm'] = 'lastfm',
	+ ['translate'] = 'translate',
	+ ['weather'] = 'weather',
	+ ['youtube'] = 'youtube',
	+ ['spotify'] = 'spotify',
	+ ['wikipedia'] = 'wikipedia',
	+ ['currency'] = 'currency',
	+ ['help'] = 'help',
	+ ['id'] = 'id',
	+ ['ban'] = 'ban',
	+ ['kick'] = 'kick',
	+ ['mute'] = 'mute',
	+ ['warn'] = 'warn',
	+ ['pin'] = 'pin',
	+ ['report'] = 'report',
	+ ['rules'] = 'rules',
	+ ['setlang'] = 'setlang',
	+ ['settings'] = 'settings',
	+}
	+
	+-- v1 config key -> .env key mapping
	+local CONFIG_KEY_MAP = {
	+ { v1 = 'bot_token', env = 'BOT_TOKEN' },
	+ { v1 = 'log_channel', env = 'LOG_CHAT' },
	+ { v1 = 'log_chat', env = 'LOG_CHAT' },
	+}
	+
	+local CONFIG_API_KEY_MAP = {
	+ { v1 = 'lastfm', env = 'LASTFM_API_KEY' },
	+ { v1 = 'youtube', env = 'YOUTUBE_API_KEY' },
	+ { v1 = 'weather', env = 'OPENWEATHERMAP_API_KEY' },
	+ { v1 = 'spotify_client_id', env = 'SPOTIFY_CLIENT_ID' },
	+ { v1 = 'spotify_client_secret', env = 'SPOTIFY_CLIENT_SECRET' },
	+ { v1 = 'spamwatch', env = 'SPAMWATCH_TOKEN' },
	+}
	+
	+-- v1 Redis key patterns (distinct from v2 patterns)
	+local V1_KEY_PATTERNS = {
	+ settings = 'chat:*:settings',
	+ welcome = 'chat:*:welcome',
	+ rules = 'chat:*:rules',
	+ warnings = 'chat::warnings:',
	+ disabled_plugins = 'chat:*:disabled_plugins',
	+ filters = 'chat:*:filters',
	+ triggers = 'chat:*:triggers',
	+ bans = 'chat:*:bans',
	+}
	+
	+-- Extract chat_id from a v1 Redis key like "chat:-100123:settings"
	+local function extract_chat_id(key)
	+ local id = key:match('^chat:(%-?%d+):')
	+ return id and tonumber(id) or nil
	+end
	+
	+-- Extract user_id from a warnings key like "chat:-100123:warnings:456"
	+local function extract_warning_user_id(key)
	+ local uid = key:match(':warnings:(%d+)$')
	+ return uid and tonumber(uid) or nil
	+end
	+
	+-- Ensure a chat row exists in PostgreSQL
	+local function ensure_chat(db, chat_id)
	+ db.call('sp_upsert_chat', { chat_id, 'Imported Chat', 'supergroup', nil })
	+end
	+
	+--- Detect a v1.5 configuration.lua file
	+-- @param path string: path to configuration.lua (default: 'configuration.lua')
	+-- @return table\|nil: parsed config table, or nil if not found/invalid
	+function migrate.detect_v1_config(path)
	+ path = path or 'configuration.lua'
	+ local f = io.open(path, 'r')
	+ if not f then return nil end
	+ f:close()
	+
	+ -- Load in a sandboxed environment (no os, io, require)
	+ local sandbox = {
	+ tonumber = tonumber,
	+ tostring = tostring,
	+ type = type,
	+ pairs = pairs,
	+ ipairs = ipairs,
	+ table = { insert = table.insert, concat = table.concat },
	+ string = { format = string.format, match = string.match },
	+ math = { floor = math.floor },
	+ }
	+ local chunk, err = loadfile(path, 't', sandbox)
	+ if not chunk then
	+ logger.warn('Failed to parse v1 config %s: %s', path, tostring(err))
	+ return nil
	+ end
	+
	+ local ok, result = pcall(chunk)
	+ if not ok then
	+ logger.warn('Failed to execute v1 config %s: %s', path, tostring(result))
	+ return nil
	+ end
	+
	+ -- Validate: must have bot_token and admins
	+ if type(result) ~= 'table' then return nil end
	+ if not result.bot_token and not result['bot_token'] then return nil end
	+ if not result.admins and not result['admins'] then return nil end
	+
	+ return result
	+end
	+
	+--- Convert a v1 config table to .env format string
	+-- @param v1_config table: parsed v1 configuration table
	+-- @return string: .env file content
	+function migrate.convert_config(v1_config)
	+ if not v1_config or type(v1_config) ~= 'table' then
	+ return ''
	+ end
	+
	+ local lines = { '# Auto-generated from v1.5 configuration.lua' }
	+
	+ -- Direct top-level mappings
	+ for _, mapping in ipairs(CONFIG_KEY_MAP) do
	+ local val = v1_config[mapping.v1]
	+ if val and tostring(val) ~= '' then
	+ table.insert(lines, mapping.env .. '=' .. tostring(val))
	+ end
	+ end
	+
	+ -- Admins (table -> comma-joined)
	+ if type(v1_config.admins) == 'table' then
	+ local admin_ids = {}
	+ for _, id in ipairs(v1_config.admins) do
	+ table.insert(admin_ids, tostring(id))
	+ end
	+ if #admin_ids > 0 then
	+ table.insert(lines, 'BOT_ADMINS=' .. table.concat(admin_ids, ','))
	+ end
	+ end
	+
	+ -- Redis config
	+ if type(v1_config.redis) == 'table' then
	+ if v1_config.redis.host and tostring(v1_config.redis.host) ~= '' then
	+ table.insert(lines, 'REDIS_HOST=' .. tostring(v1_config.redis.host))
	+ end
	+ if v1_config.redis.port then
	+ table.insert(lines, 'REDIS_PORT=' .. tostring(v1_config.redis.port))
	+ end
	+ if v1_config.redis.password and tostring(v1_config.redis.password) ~= '' then
	+ table.insert(lines, 'REDIS_PASSWORD=' .. tostring(v1_config.redis.password))
	+ end
	+ end
	+
	+ -- API keys from keys subtable
	+ if type(v1_config.keys) == 'table' then
	+ for _, mapping in ipairs(CONFIG_API_KEY_MAP) do
	+ local val = v1_config.keys[mapping.v1]
	+ if val and tostring(val) ~= '' then
	+ table.insert(lines, mapping.env .. '=' .. tostring(val))
	+ end
	+ end
	+ end
	+
	+ table.insert(lines, '')
	+ return table.concat(lines, '\n')
	+end
	+
	+--- Scan Redis for v1-era key patterns
	+-- @param redis table: Redis connection
	+-- @return table: categorized keys { settings = {...}, welcome = {...}, ... }
	+function migrate.scan_v1_keys(redis)
	+ local result = {}
	+ local total = 0
	+ for category, pattern in pairs(V1_KEY_PATTERNS) do
	+ local keys = redis.scan(pattern)
	+ result[category] = keys or {}
	+ total = total + #(keys or {})
	+ end
	+ result._total = total
	+ return result
	+end
	+
	+--- Check if v1.5 data is present
	+-- @param redis table: Redis connection
	+-- @param opts table: optional { v1_config_path = 'configuration.lua' }
	+-- @return table: { detected = bool, config_file = path\|nil, key_count = number }
	+function migrate.check(redis, opts)
	+ opts = opts or {}
	+ local config_path = opts.v1_config_path or 'configuration.lua'
	+
	+ local config_detected = false
	+ local f = io.open(config_path, 'r')
	+ if f then
	+ f:close()
	+ config_detected = true
	+ end
	+
	+ local keys = migrate.scan_v1_keys(redis)
	+ local key_count = keys._total or 0
	+
	+ return {
	+ detected = config_detected or key_count > 0,
	+ config_file = config_detected and config_path or nil,
	+ key_count = key_count,
	+ }
	+end
	+
	+--- Import chat settings from v1 Redis hashes to PostgreSQL
	+-- @param db table: database connection
	+-- @param redis table: Redis connection
	+-- @param keys table: array of v1 setting keys
	+-- @return number: count of imported records
	+function migrate.import_chat_settings(db, redis, keys)
	+ local count = 0
	+ for _, key in ipairs(keys) do
	+ local chat_id = extract_chat_id(key)
	+ if chat_id then
	+ ensure_chat(db, chat_id)
	+ local settings = redis.hgetall(key)
	+ for field, value in pairs(settings) do
	+ local mapped = SETTING_KEY_MAP[field] or field
	+ db.call('sp_upsert_chat_setting_if_missing', { chat_id, mapped, value })
	+ count = count + 1
	+ end
	+ end
	+ end
	+ return count
	+end
	+
	+--- Import welcome messages from v1 Redis strings to PostgreSQL
	+-- @param db table: database connection
	+-- @param redis table: Redis connection
	+-- @param keys table: array of v1 welcome keys
	+-- @return number: count of imported records
	+function migrate.import_welcome_messages(db, redis, keys)
	+ local count = 0
	+ for _, key in ipairs(keys) do
	+ local chat_id = extract_chat_id(key)
	+ if chat_id then
	+ local message = redis.get(key)
	+ if message and message ~= '' then
	+ ensure_chat(db, chat_id)
	+ db.call('sp_upsert_welcome_message', { chat_id, message })
	+ count = count + 1
	+ end
	+ end
	+ end
	+ return count
	+end
	+
	+--- Import rules from v1 Redis strings to PostgreSQL
	+-- @param db table: database connection
	+-- @param redis table: Redis connection
	+-- @param keys table: array of v1 rules keys
	+-- @return number: count of imported records
	+function migrate.import_rules(db, redis, keys)
	+ local count = 0
	+ for _, key in ipairs(keys) do
	+ local chat_id = extract_chat_id(key)
	+ if chat_id then
	+ local rules_text = redis.get(key)
	+ if rules_text and rules_text ~= '' then
	+ ensure_chat(db, chat_id)
	+ db.call('sp_upsert_rules', { chat_id, rules_text })
	+ count = count + 1
	+ end
	+ end
	+ end
	+ return count
	+end
	+
	+--- Import warnings from v1 Redis strings to v2 Redis hashes + PostgreSQL
	+-- @param db table: database connection
	+-- @param redis table: Redis connection
	+-- @param keys table: array of v1 warning keys
	+-- @return number: count of imported records
	+function migrate.import_warnings(db, redis, keys)
	+ local count = 0
	+ for _, key in ipairs(keys) do
	+ local chat_id = extract_chat_id(key)
	+ local user_id = extract_warning_user_id(key)
	+ if chat_id and user_id then
	+ local warn_count = tonumber(redis.get(key)) or 0
	+ if warn_count > 0 then
	+ ensure_chat(db, chat_id)
	+ -- Set v2 Redis hash (matches warn plugin pattern)
	+ local v2_key = 'chat:' .. chat_id .. ':' .. user_id
	+ redis.hset(v2_key, 'warnings', tostring(warn_count))
	+ -- Insert warning rows in DB
	+ for _ = 1, warn_count do
	+ db.call('sp_insert_warning', { chat_id, user_id, 0, 'Imported from v1.5' })
	+ end
	+ count = count + warn_count
	+ end
	+ end
	+ end
	+ return count
	+end
	+
	+--- Import disabled plugins from v1 Redis hashes to PostgreSQL + v2 Redis sets
	+-- @param db table: database connection
	+-- @param redis table: Redis connection
	+-- @param keys table: array of v1 disabled_plugins keys
	+-- @return number: count of imported records
	+function migrate.import_disabled_plugins(db, redis, keys)
	+ local count = 0
	+ for _, key in ipairs(keys) do
	+ local chat_id = extract_chat_id(key)
	+ if chat_id then
	+ ensure_chat(db, chat_id)
	+ local plugins = redis.hgetall(key)
	+ for name in pairs(plugins) do
	+ local mapped = PLUGIN_NAME_MAP[name] or name
	+ -- Insert into PostgreSQL disabled_plugins table
	+ db.execute(
	+ 'INSERT INTO disabled_plugins (chat_id, plugin_name) VALUES ($1, $2) ON CONFLICT DO NOTHING',
	+ { chat_id, mapped }
	+ )
	+ -- Also set v2 Redis set (matches session.lua pattern)
	+ redis.sadd('disabled_plugins:' .. chat_id, mapped)
	+ count = count + 1
	+ end
	+ end
	+ end
	+ return count
	+end
	+
	+--- Import filters from v1 Redis hashes to PostgreSQL
	+-- @param db table: database connection
	+-- @param redis table: Redis connection
	+-- @param keys table: array of v1 filter keys
	+-- @return number: count of imported records
	+function migrate.import_filters(db, redis, keys)
	+ local count = 0
	+ for _, key in ipairs(keys) do
	+ local chat_id = extract_chat_id(key)
	+ if chat_id then
	+ ensure_chat(db, chat_id)
	+ local filters = redis.hgetall(key)
	+ for pattern, action in pairs(filters) do
	+ db.call('sp_insert_filter', { chat_id, pattern, action, nil })
	+ count = count + 1
	+ end
	+ end
	+ end
	+ return count
	+end
	+
	+--- Import triggers from v1 Redis hashes to PostgreSQL
	+-- @param db table: database connection
	+-- @param redis table: Redis connection
	+-- @param keys table: array of v1 trigger keys
	+-- @return number: count of imported records
	+function migrate.import_triggers(db, redis, keys)
	+ local count = 0
	+ for _, key in ipairs(keys) do
	+ local chat_id = extract_chat_id(key)
	+ if chat_id then
	+ ensure_chat(db, chat_id)
	+ local triggers = redis.hgetall(key)
	+ for pattern, response in pairs(triggers) do
	+ db.call('sp_insert_trigger', { chat_id, pattern, response, nil })
	+ count = count + 1
	+ end
	+ end
	+ end
	+ return count
	+end
	+
	+--- Import bans from v1 Redis sets to PostgreSQL group_blocklist
	+-- @param db table: database connection
	+-- @param redis table: Redis connection
	+-- @param keys table: array of v1 ban keys
	+-- @return number: count of imported records
	+function migrate.import_bans(db, redis, keys)
	+ local count = 0
	+ for _, key in ipairs(keys) do
	+ local chat_id = extract_chat_id(key)
	+ if chat_id then
	+ ensure_chat(db, chat_id)
	+ local members = redis.smembers(key)
	+ for _, user_id_str in ipairs(members) do
	+ local user_id = tonumber(user_id_str)
	+ if user_id then
	+ db.call('sp_upsert_blocklist_entry', { chat_id, user_id, 'Imported from v1.5' })
	+ count = count + 1
	+ end
	+ end
	+ end
	+ end
	+ return count
	+end
	+
	+--- Clean up all v1 Redis keys after successful migration
	+-- @param redis table: Redis connection
	+-- @param keys table: categorized key map from scan_v1_keys
	+-- @return number: count of deleted keys
	+function migrate.cleanup_v1_keys(redis, keys)
	+ local count = 0
	+ for category, key_list in pairs(keys) do
	+ if category ~= '_total' then
	+ for _, key in ipairs(key_list) do
	+ redis.del(key)
	+ count = count + 1
	+ end
	+ end
	+ end
	+ return count
	+end
	+
	+--- Run the full v1.5 -> v2.x migration pipeline
	+-- @param db table: database connection
	+-- @param redis table: Redis connection
	+-- @param opts table: { dry_run = false, skip_cleanup = false, v1_config_path = 'configuration.lua' }
	+-- @return table: { success, already_migrated, config_migrated, records_imported, keys_cleaned, errors }
	+function migrate.run(db, redis, opts)
	+ opts = opts or {}
	+ local dry_run = opts.dry_run or false
	+ local skip_cleanup = opts.skip_cleanup or false
	+ local config_path = opts.v1_config_path or 'configuration.lua'
	+
	+ local result = {
	+ success = false,
	+ already_migrated = false,
	+ config_migrated = false,
	+ records_imported = 0,
	+ keys_cleaned = 0,
	+ errors = {},
	+ }
	+
	+ -- 1. Check if already migrated
	+ local applied = db.execute(
	+ 'SELECT 1 FROM schema_migrations WHERE name = $1',
	+ { 'v1_data_import' }
	+ )
	+ if applied and #applied > 0 then
	+ result.success = true
	+ result.already_migrated = true
	+ return result
	+ end
	+
	+ -- 2. Detect and convert v1 config
	+ local v1_config = migrate.detect_v1_config(config_path)
	+ if v1_config and not dry_run then
	+ local env_file = io.open('.env', 'r')
	+ if not env_file then
	+ local env_content = migrate.convert_config(v1_config)
	+ local out = io.open('.env', 'w')
	+ if out then
	+ out:write(env_content)
	+ out:close()
	+ result.config_migrated = true
	+ logger.info('Converted v1 configuration.lua to .env')
	+ end
	+ else
	+ env_file:close()
	+ logger.info('Skipping config conversion: .env already exists')
	+ end
	+ end
	+
	+ -- 3. Scan v1 Redis keys
	+ local keys = migrate.scan_v1_keys(redis)
	+ if keys._total == 0 and not v1_config then
	+ result.success = true
	+ return result
	+ end
	+
	+ if dry_run then
	+ result.success = true
	+ result.records_imported = keys._total
	+ return result
	+ end
	+
	+ -- 4-7. Run imports inside a transaction
	+ local import_count = 0
	+ local tx_ok, tx_err = pcall(function()
	+ db.query('BEGIN')
	+
	+ -- 5. Import all categories
	+ if keys.settings and #keys.settings > 0 then
	+ import_count = import_count + migrate.import_chat_settings(db, redis, keys.settings)
	+ end
	+ if keys.welcome and #keys.welcome > 0 then
	+ import_count = import_count + migrate.import_welcome_messages(db, redis, keys.welcome)
	+ end
	+ if keys.rules and #keys.rules > 0 then
	+ import_count = import_count + migrate.import_rules(db, redis, keys.rules)
	+ end
	+ if keys.warnings and #keys.warnings > 0 then
	+ import_count = import_count + migrate.import_warnings(db, redis, keys.warnings)
	+ end
	+ if keys.disabled_plugins and #keys.disabled_plugins > 0 then
	+ import_count = import_count + migrate.import_disabled_plugins(db, redis, keys.disabled_plugins)
	+ end
	+ if keys.filters and #keys.filters > 0 then
	+ import_count = import_count + migrate.import_filters(db, redis, keys.filters)
	+ end
	+ if keys.triggers and #keys.triggers > 0 then
	+ import_count = import_count + migrate.import_triggers(db, redis, keys.triggers)
	+ end
	+ if keys.bans and #keys.bans > 0 then
	+ import_count = import_count + migrate.import_bans(db, redis, keys.bans)
	+ end
	+
	+ -- 6. Record migration
	+ db.execute(
	+ 'INSERT INTO schema_migrations (name) VALUES ($1)',
	+ { 'v1_data_import' }
	+ )
	+
	+ -- 7. Commit
	+ db.query('COMMIT')
	+ end)
	+
	+ if not tx_ok then
	+ db.query('ROLLBACK')
	+ table.insert(result.errors, tostring(tx_err))
	+ logger.error('v1.5 migration failed, rolled back: %s', tostring(tx_err))
	+ return result
	+ end
	+
	+ result.records_imported = import_count
	+
	+ -- 8. Cleanup v1 keys
	+ if not skip_cleanup then
	+ result.keys_cleaned = migrate.cleanup_v1_keys(redis, keys)
	+ end
	+
	+ -- 9. Rename configuration.lua
	+ local config_exists = io.open(config_path, 'r')
	+ if config_exists then
	+ config_exists:close()
	+ os.rename(config_path, config_path .. '.v1.bak')
	+ end
	+
	+ result.success = true
	+ return result
	+end
	+
	+return migrate
	diff --git a/src/db/init.lua b/src/db/init.lua
	index 36019d1..be379ee 100644
	--- a/src/db/init.lua
	+++ b/src/db/init.lua
	@@ -1,109 +1,110 @@
	--[[
	mattata v2.0 - Migration Runner
	Runs pending SQL migrations in order, wrapped in transactions.
	Supports migrations from src/db/migrations/ AND plugin.migration fields.
	]]

	local migrations = {}

	local logger = require('src.core.logger')

	local migration_files = {
	{ name = '001_initial_schema', path = 'src.db.migrations.001_initial_schema' },
	{ name = '002_federation_tables', path = 'src.db.migrations.002_federation_tables' },
	{ name = '003_statistics_tables', path = 'src.db.migrations.003_statistics_tables' },
	{ name = '004_performance_indexes', path = 'src.db.migrations.004_performance_indexes' },
	{ name = '005_stored_procedures', path = 'src.db.migrations.005_stored_procedures' },
	- { name = '006_import_procedures', path = 'src.db.migrations.006_import_procedures' }
	+ { name = '006_import_procedures', path = 'src.db.migrations.006_import_procedures' },
	+ { name = '007_v1_import_tracking', path = 'src.db.migrations.007_v1_import_tracking' }
	}

	function migrations.run(db)
	-- Create migrations tracking table
	db.query([[
	CREATE TABLE IF NOT EXISTS schema_migrations (
	name VARCHAR(255) PRIMARY KEY,
	applied_at TIMESTAMP WITH TIME ZONE DEFAULT NOW()
	)
	]])

	-- Run each migration if not already applied
	for _, mig in ipairs(migration_files) do
	local applied = db.execute(
	'SELECT 1 FROM schema_migrations WHERE name = $1',
	{ mig.name }
	)
	if not applied or #applied == 0 then
	logger.info('Running migration: %s', mig.name)
	local ok, mod = pcall(require, mig.path)
	if ok and type(mod) == 'table' and mod.up then
	-- Wrap migration in a transaction
	local begin_ok, begin_err = db.query('BEGIN')
	if not begin_ok and begin_err then
	logger.error('Failed to begin transaction for migration %s: %s', mig.name, tostring(begin_err))
	os.exit(1)
	end

	local sql = mod.up()
	local migration_ok = true
	local migration_err = nil
	-- Split on semicolons, respecting $$-delimited blocks
	local statements = {}
	local current = ''
	local in_dollar = false
	local i = 1
	while i <= #sql do
	if sql:sub(i, i + 1) == '$$' then
	in_dollar = not in_dollar
	current = current .. '$$'
	i = i + 2
	elseif sql:sub(i, i) == ';' and not in_dollar then
	local trimmed = current:match('^%s(.-)%s$')
	if trimmed ~= '' then
	statements[#statements + 1] = trimmed
	end
	current = ''
	i = i + 1
	else
	current = current .. sql:sub(i, i)
	i = i + 1
	end
	end
	local trimmed = current:match('^%s(.-)%s$')
	if trimmed ~= '' then
	statements[#statements + 1] = trimmed
	end
	for _, statement in ipairs(statements) do
	local result, err = db.query(statement)
	if not result and err then
	migration_ok = false
	migration_err = err
	break
	end
	end

	if not migration_ok then
	logger.error('Migration %s failed: %s — rolling back', mig.name, tostring(migration_err))
	db.query('ROLLBACK')
	os.exit(1)
	end

	-- Record migration as applied using parameterized query
	db.execute(
	'INSERT INTO schema_migrations (name) VALUES ($1)',
	{ mig.name }
	)
	db.query('COMMIT')
	logger.info('Migration %s applied successfully', mig.name)
	else
	logger.error('Failed to load migration %s: %s', mig.name, tostring(mod))
	os.exit(1)
	end
	else
	logger.debug('Migration %s already applied', mig.name)
	end
	end

	logger.info('All migrations up to date')
	end

	return migrations
	diff --git a/src/db/migrations/007_v1_import_tracking.lua b/src/db/migrations/007_v1_import_tracking.lua
	new file mode 100644
	index 0000000..be7b3e7
	--- /dev/null
	+++ b/src/db/migrations/007_v1_import_tracking.lua
	@@ -0,0 +1,24 @@
	+--[[
	+ Migration 007 - v1 Import Tracking
	+ Adds a helper stored procedure for bulk-importing chat settings
	+ during v1.5 -> v2.x data migration. Uses ON CONFLICT DO NOTHING
	+ to preserve any existing v2 settings.
	+]]
	+
	+local migration = {}
	+
	+function migration.up()
	+ return [[
	+
	+CREATE OR REPLACE FUNCTION sp_upsert_chat_setting_if_missing(
	+ p_chat_id BIGINT, p_key TEXT, p_value TEXT
	+) RETURNS void AS $$
	+ INSERT INTO chat_settings (chat_id, key, value)
	+ VALUES (p_chat_id, p_key, p_value)
	+ ON CONFLICT (chat_id, key) DO NOTHING;
	+$$ LANGUAGE sql;
	+
	+ ]]
	+end
	+
	+return migration

File Metadata

Mime Type: text/x-diff
Expires: Wed, Apr 1, 11:16 AM (1 d, 18 h)
Storage Engine: blob
Storage Format: Raw Data
Storage Handle: 57121
Default Alt Text: (60 KB)

No OneTemporaryActions

View Options

File Metadata

Event Timeline

No OneTemporary
Actions