about summary refs log tree commit diff
diff options
context:
space:
mode:
authorJune McEnroe <june@causal.agency>2020-01-14 17:57:04 -0500
committerJune McEnroe <june@causal.agency>2020-01-14 17:57:04 -0500
commit1140dfdd3d6bbcadb0ccb1693b4fbe4e6d0792c5 (patch)
tree804d2897a5be3c12b50d5e723f40aeadc5e8fdb5
parentAdd option for client cert and SASL EXTERNAL (diff)
downloadlitterbox-1140dfdd3d6bbcadb0ccb1693b4fbe4e6d0792c5.tar.gz
litterbox-1140dfdd3d6bbcadb0ccb1693b4fbe4e6d0792c5.zip
Add columnsize = 0 option to FTS index
Since we don't use ranking functions and I don't see them being useful,
there is no point in having columnsize, which just takes extra space in
the database.

In my database of approximately 3.5 million events, disabling columnsize
saves about 62 MB.

The migration unfortunately has to rebuild the entire index to disable
it.
Diffstat (limited to '')
-rw-r--r--database.h25
1 files changed, 22 insertions, 3 deletions
diff --git a/database.h b/database.h
index d44bb15..83be73c 100644
--- a/database.h
+++ b/database.h
@@ -31,7 +31,7 @@
 
 #define DATABASE_PATH "litterbox/litterbox.sqlite"
 
-enum { DatabaseVersion = 0 };
+enum { DatabaseVersion = 1 };
 
 #define ENUM_TYPE \
 	X(Privmsg, "privmsg") \
@@ -285,7 +285,8 @@ static const char *InitSQL = SQL(
 		network, channel, query, nick, user, target, message,
 		content = text,
 		content_rowid = event,
-		tokenize = 'porter'
+		tokenize = 'porter',
+		columnsize = 0
 	);
 
 	CREATE TRIGGER eventsInsert AFTER INSERT ON events BEGIN
@@ -300,6 +301,8 @@ static const char *InitSQL = SQL(
 		) SELECT 'delete', * FROM text WHERE event = old.event;
 	END;
 
+	PRAGMA user_version = 1;
+
 	COMMIT TRANSACTION;
 );
 
@@ -308,7 +311,23 @@ static inline void dbInit(void) {
 }
 
 static const char *MigrationSQL[] = {
-	NULL,
+	// Added columnsize = 0 option.
+	SQL(
+		BEGIN TRANSACTION;
+		DROP TABLE search;
+		CREATE VIRTUAL TABLE search USING fts5 (
+			network, channel, query, nick, user, target, message,
+			content = text,
+			content_rowid = event,
+			tokenize = 'porter',
+			columnsize = 0
+		);
+		INSERT INTO search (
+			rowid, network, channel, query, nick, user, target, message
+		) SELECT * FROM text;
+		PRAGMA user_version = 1;
+		COMMIT TRANSACTION;
+	),
 };
 
 static inline void dbMigrate(void) {
d I'm just a photographer. It's visual art. 2024-09-23Automatically select the last used lens for a bodyJune McEnroe 2024-09-19Add photos from September 5June McEnroe Had to prefix the folder number onto these file names manually because they must have come out of a different scanner or something. 2024-09-15Add some more film stocks to the listJune McEnroe 2024-09-13Add photos from September 2June McEnroe 2024-09-13Add Fomapan 200 to films listJune McEnroe 2024-09-10Add August 29 picnic photosJune McEnroe 2024-09-08Apply some bold to trips renderingJune McEnroe This seems easier to visually scan. The only other thing I'd like is a nicer date rendering but JavaScript is useless for that. 2024-09-08Render trips hopefully more efficientlyJune McEnroe 2024-09-08Allow removing bodies and lensesJune McEnroe 2024-09-08Limit body width so it looks less silly on desktopJune McEnroe 2024-09-07Handle no film being loadedJune McEnroe 2024-09-07Fancy up the text a littleJune McEnroe