stz184
642afe4966
scanner_user.go uses hashes now
2020-06-24 13:52:40 +03:00
stz184
e70da6bb26
Merge remote-tracking branch 'upstream/master' into use-hash-instead-of-full-path-to-avoid-key-length-maximum
2020-06-24 13:42:43 +03:00
stz184
f88fa33e5f
InnoDB has a maximum index length of 767 bytes for tables that use COMPACT or REDUNDANT row format, so for utf8mb3 or utf8mb4 columns, you can index a maximum of 255 or 191 characters, respectively.
...
That's why I created a new column storing a MD5 hash of the path and made it unique. The MD5 hash has only 32 characters and can be generated natively in MySQL and MariaDB. It helps us to avoid maximum key length and in the same time enforce unique photo and album paths.
- Added path_hash column to photo and album tables
- Added down migration file for 005_utf8_migration (just for consistency)
- Added PathHash field to Album and Photo struct
- album_scanner.go and photo_scanner.goo perform checks by MD5 hash now
2020-06-24 11:43:20 +03:00
viktorstrate
fba41918f4
Fix formatting error
2020-06-23 17:02:44 +02:00
viktorstrate
66a4288ec9
Throttle queue notifications
2020-06-23 16:46:38 +02:00
viktorstrate
23461ee707
Fix bugs and improvements
...
- Fix a deadlock in the scanner cache
- Cleanup error messages and logs
2020-06-23 16:42:02 +02:00
viktorstrate
e57a6f3dc0
Make sure not only new photos are scanned
...
Fix a small bug where a photo was marked as processed when it wasn't
2020-06-23 15:23:20 +02:00
viktorstrate
11c3a92373
Reintregrate notifications with scanner
2020-06-23 15:13:07 +02:00
viktorstrate
cc1eadaef4
Fix queue tests
2020-06-23 14:10:03 +02:00
viktorstrate
95f4df4b7d
Start to integrate new scanner system with api
2020-06-23 00:40:47 +02:00
viktorstrate
a22d002146
Major rework of scanner
2020-06-22 23:52:41 +02:00
viktorstrate
666ecef4b3
Refactor scanner
...
- Move scanner cache to its own file
- Move album scanner related code to its own file
2020-06-22 21:52:53 +02:00
viktorstrate
2f4212eeab
Start on scanner queue + add some tests
2020-06-22 18:02:21 +02:00
viktorstrate
d213a42869
Add supported raw formats by darktable
2020-05-17 22:32:31 +02:00
viktorstrate
b077f77c89
Make darktable run in parallel
...
by using different config directories for each photo
2020-05-17 21:49:29 +02:00
viktorstrate
14f6f8f000
Fix error when encoding raw photos
...
where path contains spaces
2020-05-17 21:28:42 +02:00
viktorstrate
ad7ff8d39d
Fix executable worker arguments
2020-05-17 16:08:58 +02:00
viktorstrate
40265a18f7
Integrate executable worker, still working on darktable
2020-05-15 16:36:02 +02:00
viktorstrate
76ade24041
start on executable worker
2020-05-15 15:23:21 +02:00
viktorstrate
0754e600b7
Make EncodeImageJPEG a method of EncodeImageData
2020-05-14 15:17:23 +02:00
viktorstrate
a4b37a392f
Split up process_photo into encode + process
2020-05-14 14:35:08 +02:00
viktorstrate
4d57d43011
Determine image type by file extension
2020-05-13 15:28:50 +02:00
viktorstrate
5a5c06fb32
Extract filetypes into its own type
2020-05-13 15:05:02 +02:00
viktorstrate
43f5d85078
Don't show an error when scanning an empty file
...
Reference #32
2020-04-17 13:15:31 +02:00
viktorstrate
d5227a4ce3
Make sure that photo directory for user exists
...
If user directory doesn't exist anymore report an error notification.
2020-04-16 11:23:34 +02:00
viktorstrate
6e420dff41
Recover if EXIF decoding panics ( #33 )
...
This fixes #33 where a curtain image would cause the goexif2 library to
panic.
Now instead of crashing the entire program, it recovers and instead
prints an error to the console and ignores the EXIF data for that image.
2020-04-16 11:02:33 +02:00
viktorstrate
afb0428c2d
Add username to scanner notifications
2020-04-15 11:27:34 +02:00
viktorstrate
b254f3abaa
Implement ScanAll resolver
2020-04-15 11:21:16 +02:00
viktorstrate
4e388c430c
Fix image EXIF orientation bug
...
Fix a bug where an image couldn't be scanned if the EXIF data were
missing the orientation field.
This resolves #29
2020-04-15 10:58:10 +02:00
viktorstrate
7b3f016a0f
Add replace debounce with throttle
...
Update dependencies
2020-03-12 13:26:11 +01:00
viktorstrate
bdec205031
Cleanup + debounce scanner notifications
2020-03-12 12:55:53 +01:00
viktorstrate
179ec5283b
Implement proper photo rotation based on exif data
2020-03-12 12:30:55 +01:00
viktorstrate
5b5ac5bbbd
Scanner - delete missing photos from database
2020-03-10 18:30:23 +01:00
viktorstrate
ebe1220520
Skip hidden folders, this closes #13
2020-03-07 15:34:32 +01:00
viktorstrate
5cee5d8dea
Add TIFF support, this closes #3
2020-03-02 16:32:24 +01:00
viktorstrate
67fabd1de4
Fix graphql query for share pages
...
Properly close rows in scanner
2020-02-29 09:21:41 +01:00
viktorstrate
2f97dcff0c
Improve scanner reliability
2020-02-27 16:26:53 +01:00
viktorstrate
458b6fb49c
Improve scanner - first scan, then process
2020-02-26 21:23:13 +01:00
viktorstrate
440814564c
Improve notifications
2020-02-26 19:44:47 +01:00
viktorstrate
96ec3663b1
Continue on exif parser error
2020-02-24 23:45:02 +01:00
viktorstrate
b10c607f3a
Add exif parsing
2020-02-24 23:30:08 +01:00
viktorstrate
d27c20e041
Improve stability of scanner
2020-02-23 18:00:08 +01:00
viktorstrate
39f68e4d8a
Support PHOTO_CACHE env variable
2020-02-23 12:43:45 +01:00
viktorstrate
1e92432b07
Make scanner proceed on error
2020-02-23 11:59:57 +01:00
viktorstrate
37aea1fb3a
Improve logging
2020-02-21 21:00:40 +01:00
viktorstrate
935dc5b55e
Work on notifications + custom logger
2020-02-21 20:51:50 +01:00
viktorstrate
64e702790f
Make scanner generate high-res jpg
...
If original image format is not supported in browser
2020-02-14 13:31:44 +01:00
viktorstrate
9361e85397
Cache scanned album directories
2020-02-12 18:45:58 +01:00
viktorstrate
4af6be17d8
Improve scanner cleanup to take user into account
2020-02-12 18:19:16 +01:00
viktorstrate
a2d0509cad
Update scanner to delete removed files
2020-02-12 18:10:52 +01:00
viktorstrate
d50a15be90
Work on scanner + add launch.json for vscode
2020-02-11 23:35:35 +01:00
viktorstrate
6e72caf2f0
Improve integration with ui
2020-02-10 12:05:58 +01:00
viktorstrate
a3a4dda286
Work on share tokens
2020-02-09 21:25:33 +01:00
viktorstrate
e6593ee7f3
Working on integrating backend with ui
2020-02-09 15:26:59 +01:00
viktorstrate
d50b8034d1
Work on photo serving and processing
2020-02-09 14:21:53 +01:00
viktorstrate
9a8701ecd0
Improve image processing
2020-02-09 12:53:21 +01:00
viktorstrate
9cf455f699
Add support for cr2 raw format
2020-02-02 21:44:09 +01:00
viktorstrate
11896a246e
Work on image processing
2020-02-02 18:18:38 +01:00
viktorstrate
d81e945b68
Start on image proccessing
2020-02-02 00:29:42 +01:00
viktorstrate
a57c111630
Work on scanner
2020-02-01 17:58:45 +01:00