1
Fork 0
Commit Graph

41 Commits

Author SHA1 Message Date
Konstantin b2d591bd1b
Don't stop scanning album on media fail (#892)
* Don't stop scanning album on media fail

* Update api/scanner/scanner_album.go

accepting suggestion from Jordan

Co-authored-by: Jordan Hellier <13520761+jordy2254@users.noreply.github.com>

---------

Co-authored-by: Konstantin Koval <kkb@ukr.net>
Co-authored-by: Jordan Hellier <13520761+jordy2254@users.noreply.github.com>
2024-03-30 11:04:03 +02:00
Lajos Koszti 1f8664d7f8
fix transaction already commited error
I encountered with the following error:

> 2023/02/05 07:33:00 /app/scanner/face_detection/face_detector.go:92 sql: transaction has already been committed or rolled back
> [0.042ms] [rows:0] SELECT * FROM `media` WHERE `media`.`id` = 823 ORDER BY `media`.`id` LIMIT 1
> 2023/02/05 07:33:00 ERROR: Error detecting faces in image (/photos/Borzsony2017/DSC_0028.NEF): sql: transaction has already been committed or rolled back

It turned out it comes from the api/routes/photos.go

I found a very similar code in album_scanner.go.
The difference I saw was that while in the single photo request the
transaction passed to the `scanner_tasks.Tasks.BeforeProcessMedia` call,
in the album_scann.go the transaction created after this call and
created from the context which returned by `BeforeProcessMedia`.

Another difference was that in the `ProcessSingleMedia` call the
`AfterProcessMedia` call was called with the same - db transaction -
context, in the album_scanner it was called outside of the transaction.

I changed the logic by merging the two behavior:
Create the transaction from the context of `BeforeProcessMedia` and also
use the transaction context in the `AfterProcessMedia`.
After the change the error disappeared.

So to have it in a common place I extracted that logic into a function
and use for both the single photo request and in the album scanner.

I did not go more deeper to find out what's going on with the context
under the hood.
2023-02-05 09:05:05 +01:00
viktorstrate a5d152f0c0
Make scanner tasks share ctx 2022-07-07 22:00:05 +02:00
viktorstrate 447f05185f
Fix scanner queue tests 2022-03-28 18:43:00 +02:00
viktorstrate b09d32019e
Make it compile again 2022-03-02 17:26:06 +01:00
viktorstrate 6e2a64bc77
WIP: split up scanner into separate tasks 2022-02-15 17:22:41 +01:00
viktorstrate 6361df1793
Start on big scanner code refactoring 2022-02-14 23:57:45 +01:00
viktorstrate 9bcfdbb2cd
Add environment variables to disable features
Face detection still needs to be hidden on the frontend if it's disabled
2021-08-31 11:34:55 +02:00
Marco Herrn 0129cb7703 Treat symlinks to directories like directories
This allows symlinking to create additional subalbums.

Closes: #431
2021-07-16 13:48:29 +02:00
viktorstrate b022a13e55
Batch exiftool runs + scanner package restructuring 2021-05-06 22:01:48 +02:00
viktorstrate 542e5c8532
Improve scanner_album NewRootPath tests 2021-04-26 20:37:29 +02:00
viktorstrate 253f126aaf
Add scanner_album tests 2021-04-26 15:07:06 +02:00
viktorstrate d03923992c
clean up api log prints 2021-04-26 10:26:54 +02:00
viktorstrate a14b12b8d4
Fix bug where DetectFaces would be called with the wrong media.
This happend because the go routine did not capture the media variable,
and so the it would change before the go routine could start
and call the DetectFaces function.
2021-03-16 22:27:27 +01:00
viktorstrate 3ae92086cd
Pass database as argument to individual face_detector functions
This allows the face detector to use transactions,
such that faces can be detected on media that has not
been fully commited yet.

This solves #214
2021-03-16 18:26:51 +01:00
Kjeldgaard 8afff70d9a Store ignore data in cache
Store album ignore data in cache and not database. Fix albums with only
ignore-media-only not to be shown in album list. Use
github.com/sabhiram/go-gitignore exclusively for photoviewignore.

Signed-off-by: Kjeldgaard <Kjeldgaard@users.noreply.github.com>
2021-03-10 21:51:36 +01:00
Kjeldgaard 3ad12e3025 Ignore media files
Added feature to ignore specific media files. E.g. to ignore .png media
files, add '*.png' to .photoviewignore file or add 'img_1234.jpg' to
ignore a specific file. This applies to the current dir and any
potential subdir. Matching is case sensitive. File ignore pattern must
include a '.', otherwise the pattern is treated as a dir ignore pattern.

Signed-off-by: Kjeldgaard <Kjeldgaard@users.noreply.github.com>
2021-02-27 00:19:03 +01:00
viktorstrate abb80ae425
Start on face detection 2021-02-15 17:35:28 +01:00
viktorstrate fcdb5b4e7b
Work on sqlite support 2021-01-17 12:45:23 +01:00
viktorstrate d78cef300a
Finish add/remove root paths 2020-12-31 00:37:11 +01:00
viktorstrate aeb05bca49
get initial scanner up and running 2020-12-22 01:14:43 +01:00
viktorstrate 907894dda3
Merge branch 'master' into v2.0.0 2020-12-17 23:18:00 +01:00
viktorstrate 4cd822f068
Update gomod to reflect repo transfer 2020-12-17 22:51:43 +01:00
viktorstrate 48c3ed93f2
Fix api tests 2020-12-14 10:41:00 +01:00
viktorstrate 4f370deab9
Merge remote-tracking branch 'origin/master' into v2.0.0 2020-12-10 12:30:10 +01:00
Vladimir Ivanov 6adc79001c
Photo duplication detection (#148)
* Fixes viktorstrate/photoview#8
- Added new property CounterpartPath to Media struct to hold the path to the counterpart JPEG file (if any)
- Added new MediaType method isBasicSupportedisBasicTypeSupported()
- Added new function isFileExists() to minimize the code duplication

* Fixes viktorstrate/photoview#8
- Chaned CounterpartPath definition from string to *string
- Added new helper method FileExtensions()
- Simplified the logic inside scanForRawCounterpartFile() and scanForCompressedCounterpartFile() functions, reducing the code duplication

* Fixes viktorstrate/photoview#8
- Added debug to fileExists() function

* Cleanup fileExists logging

Co-authored-by: viktorstrate <viktorstrate@gmail.com>
2020-12-09 11:40:37 +01:00
viktorstrate 98f13d76e6
Replace database, work on scanning 2020-11-30 21:29:49 +01:00
viktorstrate 3c23f82330 Replace database, mostly album related 2020-11-23 19:59:01 +01:00
viktorstrate 27b2f95a7b Start on migrating database integration to gorm 2020-11-23 19:39:44 +01:00
stz184 e89c84f45f When a photo is deleted or moved on the file system, the coresponding cache folder and database record are deleted.
fixes #56
2020-07-24 20:50:17 +03:00
viktorstrate 990a592fcc Expose media_type to graphql
- Make video thumbnail accessible from graphql
2020-07-11 14:21:10 +02:00
viktorstrate 9e5480188b add media_type column to media table in database 2020-07-11 14:05:06 +02:00
viktorstrate 0ab6048151 Fix scanner 2020-07-10 18:57:27 +02:00
viktorstrate df80802cab Huge refactor: rename photo to media
To prepare for video support

Migrate database rename tables and columns:
- photo to media
- photo_url to media_url
- photo_exif to media_exif

- Update api accordingly
2020-07-10 14:26:19 +02:00
viktorstrate 3564866f41 Start on video processing 2020-07-10 12:58:11 +02:00
viktorstrate 23461ee707 Fix bugs and improvements
- Fix a deadlock in the scanner cache
- Cleanup error messages and logs
2020-06-23 16:42:02 +02:00
viktorstrate e57a6f3dc0 Make sure not only new photos are scanned
Fix a small bug where a photo was marked as processed when it wasn't
2020-06-23 15:23:20 +02:00
viktorstrate 11c3a92373 Reintregrate notifications with scanner 2020-06-23 15:13:07 +02:00
viktorstrate 95f4df4b7d Start to integrate new scanner system with api 2020-06-23 00:40:47 +02:00
viktorstrate a22d002146 Major rework of scanner 2020-06-22 23:52:41 +02:00
viktorstrate 666ecef4b3 Refactor scanner
- Move scanner cache to its own file
- Move album scanner related code to its own file
2020-06-22 21:52:53 +02:00