2011-12-15 Martyn Russell Release 0.13.0 2011-12-15 Aleksander Morgado libtracker-miner: fix double reference libtracker-miner: several documentation related fixes libtracker-miner: ignore next update is deprecated Fix deprecation comment and include the method between guards. libtracker-miner: document indexing tree section libtracker-miner: fix enums related documentation libtracker-miner: fix deprecation comment format libtracker-miner: fix type referenced in docs 2011-12-15 Martyn Russell libtracker-extract: Fix mockup example which unrefs TrackerExtractInfo->GFile This is not returned as a new reference. libtracker-common: Fix distcheck failure with new media art handling 2011-12-15 Aleksander Morgado docs: additional documentation fixes in libtracker-miner and libtracker-extract 2011-12-15 Martyn Russell libtracker-sparql: Fix the documentation in preparation for 0.13.x release libtracker-extract: Fix the documentation in preparation for 0.13.x release libtracker-miner: Fix the documentation in preparation for 0.13.x release build: Clean up more compiler warnings 2011-12-15 Aleksander Morgado build: ignore built files in git build: remove glib-based FTS parser Fixes GB#666232 2011-12-15 Jürg Billeter Remove remaining g_thread_init calls Calling g_thread_init with GLib 2.31 requires explicit dependency on gthread-2.0. However, g_type_init initializes threads since GLib 2.24, so it is no longer needed. 2011-12-14 Martyn Russell libtracker-common: Renamed albumart test to media-art build: Fixed various compiler warnings across the entire code base libtracker-common: Fixed unit tests with renamed albumart Merge branch 'media-art-enhancements' 2011-12-14 Sam Thursfield tracker-extract: Use enum to represent media-art image priorities tracker-extract: Style fixes and optimisations in tracker-media-art.c tracker-extract: Rename tracker_extract_find_external() tracker_extract_find_external() => tracker_extract_process_external_images() libtracker-miner: Rename media art update functions tracker_media_art_remove_add() => tracker_media_art_queue_removal() tracker_media_art_check_clean() => tracker_media_art_execute_queue() tracker-extract-gstreamer: Remove explicit zeroing after g_slice_new0() tracker-extract-gstreamer: Move media art parameters to MetadataExtractor This allows slightly more efficient processing because the apply_* functions can access media_art_type libtracker-common: Refactor tracker_media_art_get_path() libtracker-common: Fix media art paths when artist is not known Correct behaviour is prefix-md5sum(title)-md5sum( ) but previously we were generating prefix-md5sum( )-md5sum(title) tracker-extract: Fix whitespace errors tracker-extract-gstreamer: Fix compile warnings libtracker-extract,tracker-extract: Return guaranteed title When a title is generated from the filename (--enable-guarantee-metadata), return it to the calling extractor so that it can be used for media art. This allows storing video poster art for any video, provided guaranteed metadata is enabled. Rename albumart functions to media-art We now support video posters as well! Fixes GB#660784 2011-12-14 Aleksander Morgado tracker-extract,text: try to extract text from files in typical windows charsets If the input file is not valid UTF-8, we now try: * UTF-16, if NUL bytes are found in the string (windows-1252 and locale encodings are not expected to have NUL bytes within the string). * If locale encoding is not UTF-8, try with the locale encoding. * If locale encoding didn't help, try with windows-1252. Fixes GB#655383. 2011-12-14 Sam Thursfield tracker-extract: Rewrite tracker_albumart_find_external() The new code reads through the directory in one pass and allows more detailed logic to run on the results, also taking into account the type of media file. tracker-extract: Minor albumart code cleanups tracker-extract: Extract tracker_albumart_find_external() This is code from the albumart_heuristic() function but extracted to be testable and more flexible. tracker-extract: Rework albumart API to allow multiple media types * tracker-extract-gstreamer, tracker-extract-mp3: Update to use new API * tracker-extract-mp3: Pass URI instead of filename for consistency 2011-12-14 Martyn Russell Merge branch 'gsettings-keyfile-bridge' libtracker-common: Document the TRACKER_USE_CONFIG_FILES env var For tracker-extract, tracker-miner-fs, tracker-store and in the reference for env vars. 2011-12-13 Carlos Garnacho libtracker-miner: honor TRACKER_DIRECTORY_FLAG_CHECK_MTIME TrackerFileNotifier won't do mtime checks for files coming from initial crawling, if the check is result of a monitor event, it will be performed anyway. tracker-miner-fs: respect the crawling-interval setting mtime checks will only be performed on configuration directories according to this setting. tracker-miner-fs: Don't add directories to index too eagerly ensure_mount_point_exists() is expected to create the sparql for a mount point directory, not having it added to the TrackerIndexingTree. libtracker-miner: Use interned GFiles in TrackerIndexingTree signals This is so callers may rely on equality instead of g_file_equal(), fixes cancel-crawling-on-unmount in TrackerFileNotifier. 2011-12-13 Jürg Billeter tracker-store: Fix memory leak in Steroids.update This is caused by a bug in valac. libtracker-common: Fix memory leak in tracker_string_to_date Fixes NB#294705. libtracker-data: Fix small memory leak gvdb: Merge upstream changes This fixes some memory leaks. 2011-12-12 Aleksander Morgado libtracker-common: if testing old .cfg files, ensure one is always available Create the .cfg file when we detect that it doesn't exist. 2011-12-12 Carlos Garnacho libtracker-common: Add a way to test things with old .cfg files The TRACKER_USE_CONFIG_FILES envvar now triggers a mode where .cfg files are dumped into GSettings, but not deleted. The changes done to GSettings are also dumped to the .cfg file after any g_settings_apply() call with unapplied data. 2011-12-09 Carlos Garnacho tracker-extract: Don't lock on freed mutexes report_statistics() was locking on a mutex that was already freed, so actually free it after this last use. libtracker-miner: Lower the expectations in TrackerFileNotifier sparql queries Do not query for nfo:FileDataObjects, sparql preemptively added by applications could not be that complete, turning into inaccurate results. So settle for with nie:DataObject, which is a lower common denominator. 2011-12-09 Mario Blättermann [l10n] Updated German translation 2011-12-08 Matej Urbančič Updated Slovenian translation 2011-12-08 Martyn Russell tracker-miner-fs: Use new API instead of helper API to add indexing tree No actual change to functionality. Just avoid using the old API. tracker-miner-fs: Update userguide miner to work with --enable-meego Due to the miner-fs refactor, some of the libtracker-miner API has changed. 2011-12-08 Jürg Billeter libtracker-data: Fix crash due to overflow in journal reader Fixes GB#664833. 2011-12-07 Martyn Russell Merge branch 'miner-fs-refactor' build: Don't require miner-rss and miner-flickr to distcheck 2011-12-07 Carlos Garnacho libtracker-miner: Check the ignored flag in tracker_indexing_tree_file_is_indexable() It should obviously be non-indexable if its config root has that flag set. tracker-store: Finalize the TrackerDBusRequests on blank updates This was likely to leave stuck resources for already disconnected clients in the tracker-store side. tests: Add TrackerFileNotifier file monitoring tests tests: Add tests for TrackerFileNotifier configuration changes libtracker-miner: Handle directories with IGNORED flag being removed In this case, the directory could be reindexed if it's in another indexed location. libtracker-miner: Initialize TrackerDirectoryFlags to NONE in any case This is so calling tracker_indexing_tree_get_root() on an unindexed file doesn't end up with uninitialized memory as flags. libtracker-miner: Set indexing tree in the file monitor This is so monitor events on unhandled files is blocked early. libtracker-miner: issue critical warning on unlikely case This shouldn't ever happen, but issue a warning in case some bug slips in. libtracker-miner: Fix gir generation warnings libtracker-miner: Fix possibly wrong progress reporting There's a slim window between the miner calling ::process-file on the last remaining file and process_stop() where the progress would be wrongly reported as 0% libtracker-miner: Allow GFiles to be part of several TrackerFileSystem This could be needed in the future. libtracker-miner: Avoid frequent sync calls TrackerCrawler now is able to retrieve GFileInfos with a given set of attributes, avoiding the need in TrackerFileNotifier to g_file_query_info() when traversing the file tree returned on ::directory-crawled. tests: Add beginning of unit test for TrackerFileNotifier ATM only crawling is tested, file monitoring and changing configurations should be eventually tested as well libtracker-miner: Add back the code removing thumbnail/albumart The mimetype isn't needed there after all. libtracker-miner: rephrase code comment It'd be indeed good to have tracker-miner-fs monitor directories triggering a TRACKER_FILTER_PARENT_DIRECTORY filter in case that situation changes (ie. the file triggering the filter disappears), but given Tracker has never behaved like that and noone ever complained, lower the comment severity a bit, it's not something we should rush on. libtracker-miner: Remove useless code This branch of code would never be executed in practice, as TrackerFileNotifier can't ever emit ::item-moved on an unknown file. libtracker-miner: handle moving dirs from recursive to non-recursive locations TrackerMinerFS now checks the TrackerDirectoryFlags for source/dest locations, triggering a bulk delete operation on the directory children if it's moved to a non recursive location libtracker-miner: Traverse children selectively in TrackerFileSystem Now, returning FALSE in the TrackerFileSystemTraverseFunc given to tracker_file_system_traverse() means the node will not be recursed, but the traversal will still continue. This deviates a bit from g_node_traverse, but makes more sense given the common operations in a filesystem representation. This is now used in TrackerMinerFS to stop traversal on deleted folders. libtracker-miner: Avoid critical warning The extraction timer could be unset if the miner doesn't get to process any file between start/stop. libtracker-miner: Adapt commit e60fc84e6 to TrackerFileNotifier The mtimes are now stored as guint64, and a 2 seconds tolerance is used to determine whether the file changed, so lack of precision doesn't trigger spurious updates in FAT filesystems. libtracker-miner: Calculate remaining time purely on extraction time During crawling, the extraction queue may be mostly empty, but adding up time that makes the remaining time calculations to be quite off when little items have been processed so far. So, keep a timer only for extraction time, which can be used for more accurate measurements, as only extraction is left when tracker-miner-fs gets to report progress/remaining time. libtracker-miner, notifier: report again status when crawling This brings back the "Crawling directory '...'" messages reported via the Status DBus property. libtracker-miner: Remove fs->priv->is_crawling This is replaced by tracker_file_notifier_is_active() libtracker-miner, notifier: Check dir roots with the canonical copy in TrackerFileSystem libtracker-miner: Remove deleted folders from being tracked Directories are now removed from the TrackerFileSystem if a delete monitor event happens on a dir. libtracker-miner: Optimize TrackerFileSystem insertions/deletions We rely on the crawler to provide files in-order, so don't unnecessarily try to reparent child nodes that were added earlier. Also optimize insertion where a parent is provided. libtracker-miner: Cancel sparql query in TrackerFileNotifier when crawling stops libtracker-miner: Handle correctly crawling on just created dirs The crawler check was wrong for directories added due to a monitor created event, so ensure harder that we're dealing with a config root instead of some random dir. libtracker-miner: Do not delete files while traversing the file tree Instead, use tracker_file_system_delete_files(), which also avoids the need to store the GFileType as filesystem data. libtracker-miner: Fix problems with directories going out and into config again Remove the queried/crawled flags before possibly reindexing a dir tree for the second time (i.e. mounts). Also, check correctly the pending index roots when a dir is added or removed. tracker-miner-fs: Remove the preserve flag if a dir is removed from config We don't want to know anymore about that directory, even if it happened to be within a mount point tracker-miner-fs: Fix some TrackerDirectoryFlags passed on mounts Some flags were missing, others wrongly overwritten libtracker-miner: lower message log level Having a root directory changing flags may be a normal operation, so don't show a warning libtracker-miner: Ensure TrackerFileSystem has a meaningful file type tracker_file_system_get_file() will be called several times on the same file, so store the first meaningful GFileType gotten through that call. libtracker-miner: Add tracker_file_system_delete_files() This function recursively deletes files of a given GFileType, or every file if G_FILE_TYPE_UNKNOWN. 2011-12-07 Aleksander Morgado libtracker-miner: indentation and alignment fixes 2011-12-07 Carlos Garnacho libtracker-miner: Ensure the timer is set on ::directory-started libtracker-miner: Fix invalid read in TrackerFileSystem tracker-miner-fs: Remove commented code libtracker-miner: Untangle filesystem refcount over GFiles TrackerFileSystem actually owns a reference to the contained GFiles, it just expect an external g_object_unref() call over these to manage the associated GNode. libtracker-miner: Fix typo node_free() in tracker-indexing-tree.c is meant to be a GNodeTraverseFunc. tracker-miner-fs: Do not use tracker_miner_fs_directory_remove* Instead use tracker_indexing_tree_remove() directly. libtracker-miner: Avoid IRI queries on parents of config dirs libtracker-miner: Reimplement tracker_miner_fs_directory_remove* These functions now just call tracker_indexing_tree_remove(), every cancellation is now done in the signal handler. libtracker-miner: Make filesystem properties registration global This is so GDestroyNotify for set properties is available at the time of destructing a file node. tracker-miner-fs: Do not use old API to add/recheck dirs libtracker-miner: Add tracker_indexing_tree_list_roots() This function retrieves the configured roots, the flags can be later queried with tracker_indexing_tree_get_root() on these files. libtracker-miner: Listen to ::directory-updated in TrackerFileNotifier libtracker-miner: Add TrackerIndexingTree::directory-updated libtracker-miner: plug some leaks libtracker-miner: Remove unused vmethods from TrackerMinerFS This touches public API, but these signals aren't emitted anymore as TrackerIndexingTree provides all necessary information. libtracker-miner: Use level order when traversing the filesystem tree This is most similar to how TrackerMinerFS used to work, and results in less "miner stopped due to parent being indexed" situations than G_PRE_ORDER. tests: Fix filesystem test, TrackerFile is gone libtracker-miner: Obey "ignored" directory flag in TrackerFileNotifier libtracker-miner: Fix compiler warnings libtracker-miner: Stop crawler if it matches a config dir being removed libtracker-miner: Handle root config directories being deleted. libtracker-miner: Improve a bit reference counting on TrackerFileNotifier libtracker-miner: Add a "preserve in store" flag for config dirs And use it for mounts, so the info is preserved in the store, and only tracker:available is unset in TrackerMinerFiles. libtracker-miner: Remove indexing tree signal handlers in TrackerMinerFS tracker_miner_fs_directory_add*() is meant to use TrackerIndexingTree underneath, so don't do it the other way around too. libtracker-miner: Only emit ::file-deleted on folders for the toplevel being deleted libtracker-miner: Also check "config root within config root" during queries libtracker-miner: Improve logging of TrackerFileNotifier libtracker-miner: Add compat layer for tracker_miner_fs_directory_* All these functions use TrackerIndexingTree underneath now libtracker-miner: Remove item_query_exists() from TrackerMinerFS All its uses have been either removed (state is guaranteed within TrackerFileNotifier) or replaced by tracker_file_notifier_get_file_iri() libtracker-miner: Remove unused structs from TrackerMinerFS libtracker-miner: Remove miner_fs->priv->config_directories This is replaced by TrackerIndexingTree and TrackerDirectoryFlags libtracker-miner-fs: Add tracker_indexing_tree_file_is_root() libtracker-miner: Remove miner_fs->priv->directories All its uses have been replaced by TrackerFileNotifier libtracker-miner: Remove miner_fs->priv->crawled_directories It's not used anymore, replaced by TrackerFileNotifier. libtracker-miner: Only spare an IRI query on updated items libtracker-miner: Ensure the right processing order on QUEUE_WAIT situations Ensure the dirs we're waiting for are prepended to the queue, so they're processed before any later child. libtracker-miner: Avoid double checking root directories in TrackerFileNotifier If a directory happens to be both a root directory and the child of a root directory, it could be told to be inspected twice, only do this if this directory root itself is being inspected. libtracker-miner: Only append dirs to crawler processing on recursive dirs This saves some idle jumps when a directory is found on a non-recursive crawled dir. libtracker-miner: Remove dead code in TrackerMinerFS TrackerCrawler is not in use anymore, so remove all its signal handlers libtracker-miner: Remove IRI cache from TrackerMinerFS TrackerFileNotifier provides that information where necessary libtracker-miner: Add tracker_file_notifier_get_file_iri() This function replaces the IRI cache in TrackerMinerFS libtracker-miner: Remove TrackerFile Instead, weak refs to GFiles are kept to manage the actual nodes in a TrackerFileSystem tree, this also eases TrackerFile vs GFile situations. libtracker-miner: Hook up progress/status indication to TrackerFileNotifier processing libtracker-miner: Add tracker_file_notifier_is_active() Just a boolean getter telling whether the notifier is currently doing something libtracker-miner: Add TrackerFileNotifier::finished This signal notifies when have all pending operations finished, if anything triggers new operations, a new ::finished signal will be emitted after these (i.e. there's no first vs !first index difference) libtracker-miner: Add TrackerFileNotifier::directory-started/finished These 2 signals notify when does indexing of a config root directory, or a recently added directory, happens. libtracker-miner: Remove unused filesystem property libtracker-miner: Remove check for removed files during crawling This code is unused now, TrackerFileNotifier will issue ::file-deleted already for items found in the store but not during crawling. libtracker-miner: Remove mtime cache It's unused now that we have TrackerFileNotifier. Code has been roughly wired at places, but that code is to be removed soon too, so... libtracker-miner: Remove all trace of TrackerMonitor in TrackerMinerFS TrackerFileNotifier's is used instead libtracker-miner: Use TrackerFileNotifier in TrackerMinerFS Now TrackerMinerFS uses TrackerFileNotifier signals to receive crawler/monitor events, so now a cleanup is due here. libtracker-miner: Hook TrackerFileNotifier to TrackerMonitor signals libtracker-miner: Add TrackerFileNotifier::file-moved signal libtracker-miner: Add "attributes_only" parameter to TrackerFileNotifer::file-updated This is so we use the same signal on both file and file attributes updates libtracker-miner: Set up monitor in TrackerFileNotifier libtracker-miner: Perform parallel querying/crawling in TrackerFileNotifier The query is done in one big batch to minimize latency, when the last of both operations end, availability/mtimes are checked in order to issue the corresponding signal for a file change. libtracker-miner: Store mtime during crawling in TrackerFileNotifier libtracker-miner: Initialize TrackerFileNotifier filesystem properties libtracker-miner: Fix double inserts in tracker_file_system_get_file() If parent was provided, no check for duplicates was done. libtracker-miner: Add tracker_file_system_traverse() This function iterates over all known files, it is possible to provide a root to start from, or NULL. libtracker-miner: Create signals for TrackerFileNotifier libtracker-miner: Rename TrackerFileNotifier::file-added to file-created libtracker-miner: clear directories queue before TrackerCrawler::finished This is so we can call tracker_crawler_start() within the ::finished handler, instead of on an idle. libtracker-miner: Add TrackerFileNotifier TrackerFileNotifier is meant to be a high-level object that notifies about any update/addition/removal affecting the indexed content of a miner. At the moment only basic infrastructure has been added, so it only crawls the contents. tests: Add TrackerFileSystem test suite libtracker-miner: Add TrackerFileSystem This is a file system abstraction object which may provide canonical (and persistent) TrackerFile objects representing a file, so == and != operations can be used for these. It is also able to store arbitrary information tied to a file. tracker-miner-fs: Use filter policies in TrackerMinerApplications Now all files are rejected, unless they match the *.desktop|*.directory filter libtracker-miner: Add default policies to TrackerIndexingTree filters An "accept" policy means a file is accepted unless it matches a filter, meanwhile a "deny" policy means files are rejected unless they match a filter. The default policy is "accept" tracker-miner-fs: Make TrackerMinerApplications use TrackerIndexingTree libtracker-miner: Make TrackerMinerFS use TrackerIndexingTree No more check-* or monitor-directory signals are emitted. Instead, the TrackerIndexingTree is used to determine whether files should be indexed. libtracker-miner: Implement tracker_indexing_tree_file_is_indexable on top of get_root() libtracker-miner: Rename tracker_indexing_tree_get_effective_parent to get_root() "Effective parent" is misleading since the same file could be returned, so talking about "configured root directories" makes somewhat more sense. tracker-miner-fs: Specify the index directories to TrackerIndexingTree tracker-miner-fs: Set TrackerIndexingTree filters in TrackerMinerFiles libtracker-miner: Optionally filter hidden files in TrackerIndexingTree libtracker-miner: Fix infinite loop libtracker-miner: change arguments in tracker_indexing_tree_get_parent_is_indexable() Now a GList is passed instead of an array. libtracker-miner: Add ::directory-added/removed signals to TrackerIndexingTree 2011-12-07 Aleksander Morgado libtracker-miner, tests: ignore built files libtracker-miner, tests: implement multiple cases for indexing tree add/remove. libtracker-miner,tests: First tests to check indexing tree capabilities libtracker-miner: new IGNORE flag for the indexing tree TRACKER_DIRECTORY_FLAG_IGNORE allows specifying a subtree to be completely ignored. It has the same effect as adding a new path to the tree without the TRACKER_DIRECTORY_FLAG_MONITOR flag. libtracker-miner: fix logic to tell whether file is indexable A file is indexable if: * The exact file was added to be monitored. * The direct parent directory of the file was added to be monitored. * The closest configured parent directory was added to be recursively monitored. libtracker-miner: fix tree disposal libtracker-miner: allow priting indexing tree for debugging libtracker-miner: fix node appending libtracker-miner, tests: avoid aligning backslash in Makefile.am libtracker-miner: avoid extra file type checks libtracker-miner: overwrite flags when adding same path to the indexing tree libtracker-miner: indentation fixes 2011-12-07 Carlos Garnacho libtracker-miner: Make TrackerMonitor use TrackerIndexingTree This is at the moment only used for event discarding, and for emitting the correct signal out of a MOVE event. libtracker-miner: Add tracker_miner_fs_get_indexing_tree() The returned object is owned by the miner, and will be internally used to find out whether files should be processed. libtracker-miner: Add TrackerIndexingTree This object holds the parameters for the indexed directories, and the applying filters, so can effectively tell whether a file should be indexed or not. 2011-12-06 Daniel Mustieles Updated Spanish translation 2011-12-06 Martyn Russell libtracker-fts: Fix "enable-unaccent" description spelling mistake Fixes GB#664807. Ontology: Notify about nfo:DataObject changes Fixes GB#664070. tracker-extract-vorbis: Extract nmm:albumArtist for nmm:MusicAlbum This was missed in the Vorbis extractor but works in the GStreamer extractor. Fixes GB#664549. 2011-12-06 Philip Van Hoof plugins/evolution: Fixes for several crashes in the plugin More information here: https://bugzilla.gnome.org/show_bug.cgi?id=644695#c17 2011-12-05 Jürg Billeter libtracker-data: Handle error case when resetting collator Fixes NB#293766. 2011-12-03 Jorge González Updated Spanish translation 2011-12-02 Matej Urbančič Updated Slovenian translation 2011-12-02 Carlos Garnacho tracker-extract,gstreamer: Fix leak Append elements to extractor->tagcache instead of replacing the list. libtracker-miner: Fix flow control in sparql buffer Avoid piling up flush requests of a small number of elements when we reach a limit, so the buffer is actually filled up for the next flush. 2011-12-01 Jürg Billeter Merge branch 'subsecond' tests: Update tests for changed date/time formatting libtracker-data: Use tracker_date_to_string instead of SQLite's strftime This fixes an off by 1 error for timestamps before 1970. Fixes NB#293344. libtracker-common: Fix tracker_date_to_string for timestamps before 1970 libtracker-common: Fix invalid read in tracker_string_to_date libtracker-bus: Keep connection to D-Bus session bus alive Fixes NB#293333. functional-tests: Disable userguides miner to avoid timeouts 2011-11-29 Jürg Billeter libtracker-common: Add more checks to gconf-dbus locale handling Fixes NB#289635. 2011-11-29 Martyn Russell Merge branch 'miner-fs-disable-miners' tracker-miner-fs: Provide --disable-miner command line argument This is to be able to disable the files, applications or userguides miner from actually starting. Starting here, means crawling/monitoring/indexing. The miners are still established via D-Bus. 2011-11-28 Christian Kirbach [l10n] Updated German translation 2011-11-28 Carlos Garnacho tracker-miner-fs: Fix compilation with --disable-maemo some ifdefs were out of place. 2011-11-27 Lucian Adrian Grijincu Updated Romanian translation 2011-11-27 Jürg Billeter Remove g_thread_init calls Calling g_thread_init with GLib 2.31 requires explicit dependency on gthread-2.0. However, g_type_init initializes threads since GLib 2.24, so it is no longer needed. 2011-11-25 Marek Černocký Updated Czech translation 2011-11-25 Piotr Drąg Updated Polish translation 2011-11-25 Daniel Mustieles Updated Spanish translation 2011-11-25 Matej Urbančič Updated Slovenian translation Updated Slovenian translation