b022a13e55
Batch exiftool runs + scanner package restructuring
2021-05-06 22:01:48 +02:00
542e5c8532
Improve scanner_album NewRootPath tests
2021-04-26 20:37:29 +02:00
253f126aaf
Add scanner_album tests
2021-04-26 15:07:06 +02:00
1029b61a4c
Fix faces not getting scanned
...
- This fixes #344
- Add integration tests for face recognition
- Properly check that the user own the queried album
2021-04-26 12:21:15 +02:00
d03923992c
clean up api log prints
2021-04-26 10:26:54 +02:00
3073cfb953
Attempt to fix clear db for mysql
2021-04-24 19:28:20 +02:00
761db0639d
Add simple scanner integration test
2021-04-24 18:51:21 +02:00
60f8a68bf6
Implement database integration tests
2021-04-23 15:10:23 +02:00
dc8325bd4a
Extend exif tests
2021-04-22 22:46:52 +02:00
88a9d705a9
Add integration tests for exif parsers
2021-04-22 21:37:59 +02:00
084d9bfef7
Cleanup logs and external exif parser
2021-04-11 22:55:44 +02:00
ceb8c4103f
Fix exiftool detection
2021-04-03 21:39:32 +02:00
95c174d3a5
Setup husky hooks for api
2021-04-03 17:49:18 +02:00
3bde799fb3
Fix log bug
...
Fix log bug, exposure time is now float and not string.
Signed-off-by: Kjeldgaard <Kjeldgaard@users.noreply.github.com>
2021-03-31 23:43:44 +02:00
f946500eaa
Use barasher/go-exiftool
...
Use barasher/go-exiftool again.
Exposure is not saved as float and not a string.
Signed-off-by: Kjeldgaard <Kjeldgaard@users.noreply.github.com>
2021-03-31 23:35:25 +02:00
454b2dbadd
Merge branch 'master' into 244_exiftool
2021-03-29 12:10:02 +02:00
1193222f92
Update external exif parser method
...
Exiftool meta data parsed as intergers and not strings. Keep internal
exif parsing functionality.
Signed-off-by: Kjeldgaard <Kjeldgaard@users.noreply.github.com>
2021-03-28 23:52:51 +02:00
4344e895d1
Support .heic images + refactor EncodeMediaData
2021-03-27 21:14:42 +01:00
7b5305579b
Extract metadata as integers
...
Extract all metada as integers fomr exiftool.
Signed-off-by: Kjeldgaard <Kjeldgaard@users.noreply.github.com>
2021-03-23 21:36:03 +01:00
1f9f6ebf9a
Use exiftool for parsing exif data
...
Use exiftool to parse exif metadata.
Signed-off-by: Kjeldgaard <Kjeldgaard@users.noreply.github.com>
2021-03-20 00:26:05 +01:00
a14b12b8d4
Fix bug where DetectFaces would be called with the wrong media.
...
This happend because the go routine did not capture the media variable,
and so the it would change before the go routine could start
and call the DetectFaces function.
2021-03-16 22:27:27 +01:00
3ae92086cd
Pass database as argument to individual face_detector functions
...
This allows the face detector to use transactions,
such that faces can be detected on media that has not
been fully commited yet.
This solves #214
2021-03-16 18:26:51 +01:00
08a99195d6
Merge pull request #221 from Kjeldgaard/ignore_media
...
Ignore media files
2021-03-12 11:12:43 +01:00
8afff70d9a
Store ignore data in cache
...
Store album ignore data in cache and not database. Fix albums with only
ignore-media-only not to be shown in album list. Use
github.com/sabhiram/go-gitignore exclusively for photoviewignore.
Signed-off-by: Kjeldgaard <Kjeldgaard@users.noreply.github.com>
2021-03-10 21:51:36 +01:00
c18759e6db
Fix darktable docker install
...
and print version of executable workers on startup
2021-03-03 13:32:44 +01:00
26996b45e5
Reload faces if albums or media is deleted
2021-02-27 16:39:04 +01:00
3ad12e3025
Ignore media files
...
Added feature to ignore specific media files. E.g. to ignore .png media
files, add '*.png' to .photoviewignore file or add 'img_1234.jpg' to
ignore a specific file. This applies to the current dir and any
potential subdir. Matching is case sensitive. File ignore pattern must
include a '.', otherwise the pattern is treated as a dir ignore pattern.
Signed-off-by: Kjeldgaard <Kjeldgaard@users.noreply.github.com>
2021-02-27 00:19:03 +01:00
222a288077
V2 Ignore Directories plus trailing slash
...
Updated ignore dirs feature to allow trailing slashes in
.photoviewignore file(s).
Signed-off-by: Kjeldgaard <Kjeldgaard@users.noreply.github.com>
2021-02-23 20:57:05 +01:00
7ab9a12e43
V2 Ignore Directories
...
Looks for .photoviewignore file in all root dirs and potential subdirs.
Subdirs inherit the ignore rules from parent dir. If a .photoviewignore
file is found in a subdir, the new ignore rules are appended to the
existing ignore rules.
Ignore match engine is from 'github.com/sabhiram/go-gitignore'.
Signed-off-by: Kjeldgaard <Kjeldgaard@users.noreply.github.com>
2021-02-23 15:59:37 +01:00
f3ccc60e26
Decrease face threshold
2021-02-22 22:29:42 +01:00
20251dedd6
Add UI to move faces between face groups
2021-02-20 22:43:07 +01:00
5368b4bae0
Various improvements to faces UI
2021-02-19 23:30:43 +01:00
bdd2318afc
Implement recognizeUnlabeledFaces
2021-02-19 19:24:31 +01:00
e4df1fb706
Save face rectangles to database
2021-02-16 11:27:28 +01:00
4b0365e8aa
Implement getSamplesFromDatabase
2021-02-15 20:31:17 +01:00
abb80ae425
Start on face detection
2021-02-15 17:35:28 +01:00
557b5e22df
Check that EXIF contains date shot before comparing
...
This should fix #196
2021-02-14 15:45:58 +01:00
13f293e244
Re-encode video thumbnails if missing from cache
2021-02-14 10:30:10 +01:00
2ba8e466ef
Update media date_shot to match exif
2021-02-13 17:21:15 +01:00
214c4e8be4
Fix panic when scanning
2021-02-04 22:06:49 +01:00
f723bda7c5
Fixes photoview/issues#168
...
- Migrated mediaGeoJson resolver to GORM
- Fixed issue loading media list in MediaList() function
- Fixed typo in the column "gps_lonitude" (renamed to "gps_longitude")
2021-01-29 01:23:40 +02:00
fbebb13d28
Don't transcode web compatible video formats.
...
This relates to the following issues #131 #169 #173
2021-01-27 18:00:37 +01:00
404f1898c2
Properly add new user as owner
...
when scanning albums already owned by other users
2021-01-26 15:30:21 +01:00
9afacc3add
Fix the deleteOldUserAlbums function
2021-01-24 21:30:57 +01:00
46ae5115bb
Refactor EXIF parsing
2021-01-19 16:39:57 +01:00
107da91700
Refactor handling of environment variables
2021-01-17 16:50:48 +01:00
fcdb5b4e7b
Work on sqlite support
2021-01-17 12:45:23 +01:00
d78cef300a
Finish add/remove root paths
2020-12-31 00:37:11 +01:00
aeb05bca49
get initial scanner up and running
2020-12-22 01:14:43 +01:00
732ccd05ab
restructure user <-> albums relations in database
2020-12-20 17:10:00 +01:00
907894dda3
Merge branch 'master' into v2.0.0
2020-12-17 23:18:00 +01:00
4cd822f068
Update gomod to reflect repo transfer
2020-12-17 22:51:43 +01:00
092cbdb872
Migrate FormatSQL
2020-12-17 21:32:13 +01:00
7546987f85
Fixed issue that caused runtime error: invalid memory address or nil pointer dereference
...
because the sidecar file was trying to process before the highResMediaURL and thumbURL are actually created
2020-12-15 02:25:30 +02:00
48c3ed93f2
Fix api tests
2020-12-14 10:41:00 +01:00
4f370deab9
Merge remote-tracking branch 'origin/master' into v2.0.0
2020-12-10 12:30:10 +01:00
6adc79001c
Photo duplication detection ( #148 )
...
* Fixes viktorstrate/photoview#8
- Added new property CounterpartPath to Media struct to hold the path to the counterpart JPEG file (if any)
- Added new MediaType method isBasicSupportedisBasicTypeSupported()
- Added new function isFileExists() to minimize the code duplication
* Fixes viktorstrate/photoview#8
- Chaned CounterpartPath definition from string to *string
- Added new helper method FileExtensions()
- Simplified the logic inside scanForRawCounterpartFile() and scanForCompressedCounterpartFile() functions, reducing the code duplication
* Fixes viktorstrate/photoview#8
- Added debug to fileExists() function
* Cleanup fileExists logging
Co-authored-by: viktorstrate <viktorstrate@gmail.com>
2020-12-09 11:40:37 +01:00
f0d580116b
Improve album and media resolvers
2020-12-08 16:24:08 +01:00
4b778fbdde
Fix tests, fix photo route
2020-12-06 15:24:15 +01:00
98f13d76e6
Replace database, work on scanning
2020-11-30 21:29:49 +01:00
364521958b
Fix site_info, replace ID type in graphql queries
2020-11-30 16:42:13 +01:00
e42ac2436b
Replace database, resolve all build-time errors
2020-11-28 21:29:31 +01:00
2b966aa672
Replace database, mostly media and video
2020-11-26 20:48:04 +01:00
177ae91500
Replace database, mostly video
2020-11-25 23:06:47 +01:00
cc5d323b2a
Replace database, mostly process photo
2020-11-24 11:46:49 +01:00
2e9aa29258
Replace database, mostly media related
2020-11-23 20:43:00 +01:00
3c23f82330
Replace database, mostly album related
2020-11-23 19:59:01 +01:00
27b2f95a7b
Start on migrating database integration to gorm
2020-11-23 19:39:44 +01:00
9787fd570f
Cleanup and bugfixes
...
- Fix bug where unsupported media would be wrongly classified
- Fix bug where sidecar hash wouldn't be updated
- Replace error reporting hash with path,
to avoid potential null pointer exceptions
2020-11-22 23:42:42 +01:00
43e759339e
Update generated images on sidecar file change
2020-11-21 12:19:00 +00:00
11d67c9ae9
Add sidecar path & hash to DB
2020-11-21 11:54:01 +00:00
74581ee090
Cleanup media routes and media_name generation
2020-10-30 17:14:09 +01:00
bbff25973e
Fix bug where max_concurrent_tasks for...
...
scanner queue would be initialized to 0.
2020-09-24 17:52:53 +02:00
44a00c7415
Move setup of executable workers to function
...
- Fix startup crash if no site_info row was found
2020-09-23 14:34:26 +02:00
9c9807ecc9
Add admin setting to change max concurrent workers
2020-09-21 21:32:30 +02:00
05380354e3
Setup periodic scan background runner
2020-09-21 12:34:56 +02:00
f2493c80a5
Fix small bug where CleanupMedia...
...
would not remove old media if the album
did not contain any media at all but only sub-albums
2020-08-15 13:58:03 +02:00
8e7c43798a
Rotate images properly
...
This fixes #64 where some thumbnail images
would not have the correct orientation.
2020-08-15 12:24:54 +02:00
e0ab77b158
Use file modTime instead of changeTime...
...
to determine the date_shot value of pictures.
This value is cross-platform,and probably a
better representation for the date_shot
2020-08-14 12:20:14 +02:00
78b415871d
Add date information to media
...
This closes #61
2020-08-12 13:04:41 +02:00
cdb4089f19
Ignore hidden media files
...
This closes #57
2020-08-12 12:31:13 +02:00
849f2f25bf
Add fileSize to media_url database entries
2020-08-11 22:18:15 +02:00
b52595fe46
Save gps coordinates from exif
...
This closes #62
2020-08-11 14:46:23 +02:00
01bec379c7
Move `deleteOldUserAlbums` function to `cleanup_media.go`
2020-08-03 19:39:39 +02:00
e89c84f45f
When a photo is deleted or moved on the file system, the coresponding cache folder and database record are deleted.
...
fixes #56
2020-07-24 20:50:17 +03:00
82b18f0e9b
Cleanup + add video format
2020-07-13 18:01:31 +02:00
d681d1538c
Cleanup regarding video support
2020-07-12 14:31:50 +02:00
0e9d37ca77
Add video metadata
2020-07-12 14:17:49 +02:00
990a592fcc
Expose media_type to graphql
...
- Make video thumbnail accessible from graphql
2020-07-11 14:21:10 +02:00
9e5480188b
add media_type column to media table in database
2020-07-11 14:05:06 +02:00
0eb0319fa8
Generate video thumbnails
2020-07-11 13:39:11 +02:00
26a5c5ac29
Encode videos for web using ffmpeg
2020-07-11 13:13:31 +02:00
0ab6048151
Fix scanner
2020-07-10 18:57:27 +02:00
df80802cab
Huge refactor: rename photo to media
...
To prepare for video support
Migrate database rename tables and columns:
- photo to media
- photo_url to media_url
- photo_exif to media_exif
- Update api accordingly
2020-07-10 14:26:19 +02:00
3564866f41
Start on video processing
2020-07-10 12:58:11 +02:00
7caad82059
Add ffmpeg worker and supported video formats
2020-07-09 15:07:39 +02:00
642afe4966
scanner_user.go uses hashes now
2020-06-24 13:52:40 +03:00
e70da6bb26
Merge remote-tracking branch 'upstream/master' into use-hash-instead-of-full-path-to-avoid-key-length-maximum
2020-06-24 13:42:43 +03:00
f88fa33e5f
InnoDB has a maximum index length of 767 bytes for tables that use COMPACT or REDUNDANT row format, so for utf8mb3 or utf8mb4 columns, you can index a maximum of 255 or 191 characters, respectively.
...
That's why I created a new column storing a MD5 hash of the path and made it unique. The MD5 hash has only 32 characters and can be generated natively in MySQL and MariaDB. It helps us to avoid maximum key length and in the same time enforce unique photo and album paths.
- Added path_hash column to photo and album tables
- Added down migration file for 005_utf8_migration (just for consistency)
- Added PathHash field to Album and Photo struct
- album_scanner.go and photo_scanner.goo perform checks by MD5 hash now
2020-06-24 11:43:20 +03:00
fba41918f4
Fix formatting error
2020-06-23 17:02:44 +02:00
viktorstrate