Commit Graph

88 Commits

Author SHA1 Message Date
Nicolas Maillat
bdcbd7c0e0 Revert " "PHRAS-3215 : Revert add ThumbnailOrientation facet ; add strings"" 2022-03-08 11:49:36 +01:00
Nicolas Maillat
ccd460be27 Revert "PHRAS-3215 : add ThumbnailOrientation facet ; add strings" 2022-02-25 23:10:23 +01:00
jygaulier
406a0ce953 PHRAS-3602 : add ThumbnailOrientation facet ; add strings 2022-02-25 10:11:49 +01:00
jygaulier
e37ce3e255 [skip ci] PHRAS-3417_search-empty-fields_MASTER
add : facet for empty field

wip/todo : unit test ; conf ; translate "unset_field_facet_label"
2021-04-15 17:31:22 +02:00
jygaulier
cfd385498d - add : cache on es "tags"
- removed : variable globalstructure construction (WITH_FIELDS, WITH_FLAGS...) (was unused)
- cleanup (stopwatches etc)
2020-11-05 19:54:14 +01:00
jygaulier
5ae105d9c2 - fix : error during populate
- add : clear structure cache when structure (field or sb) is changed
2020-11-05 11:27:39 +01:00
jygaulier
4d961e2dbe - fix : added cache to es "globalstructure" construction 2020-11-04 11:35:36 +01:00
jygaulier
e36b0785ee - add some simple cache
- disable check of conf/searchengine/type
- faster construction of concept-paths
- pass strucure to es as factory (useless, to be reverted)
- add stopwatch (log into file, disabled for now)
2020-10-30 11:00:17 +01:00
Jean-Yves Gaulier
8ffeb58289 PHRAS-3001_es-geopoint-sources_4.1
WIP
- fix : gps position indexed by es now comes from the metadata fields defined in conf; (not from exif anymore).
- todo/wip : decide conf place, since today we only have an array of geo providers
- todo/wip : does exif apply as fallback if nothing in conf ? or if metadata not set in record ?
2020-03-29 20:11:03 +02:00
Jean-Yves Gaulier
20839f1dd7 return facets following the order defined in conf. 2020-02-17 20:17:08 +01:00
Jean-Yves Gaulier
53f17e1897 save modifications in conf, and also write back to databoxes 2020-02-17 20:17:08 +01:00
aynsix
df85aa8827 PHRAS-1797 porting searchengine:index to 4.1 (#2454)
* portage commande searchengine:index vers 4.1

* FIX Indexer

* Update DataboxFetcherFactory.php
2018-01-30 18:50:57 +01:00
jygaulier
a70bf2fc70 PHRAS-1304_AUTO-COMPLETION_MASTER (#2061)
* PHRAS-1304_AUTO-COMPLETION_MASTER
ported from 4.0

* PHRAS-1304_AUTO-COMPLETION_MASTER
fix

* PHRAS-1304_AUTO-COMPLETION_MASTER
fix

* PHRAS-1304_AUTO-COMPLETION_MASTER
bump php version to 5.5.31 (5.5.21 is obsolete in cicleci)

* PHRAS-1304_AUTO-COMPLETION_MASTER
bump php version to 5.5.31 : php.ini moved in circelci

* PHRAS-1304_AUTO-COMPLETION_MASTER
add zmq & date to php for circleci

* PHRAS-1304_AUTO-COMPLETION_MASTER
add zmq

* PHRAS-1304_AUTO-COMPLETION_MASTER
bump amqp

* PHRAS-1304_AUTO-COMPLETION_MASTER
downgrade amqp to 1.2 to test compilation against old librabbit 0.4 (ubuntu)

* PHRAS-1304_AUTO-COMPLETION_MASTER
add amqp.so to php.ini, (re)bump amqp to 1.6

* PHRAS-1304_AUTO-COMPLETION_MASTER
build rabittmq from git

* PHRAS-1304_AUTO-COMPLETION_MASTER
build rabittmq from git again

* PHRAS-1304_AUTO-COMPLETION_MASTER
build rabittmq from git again and again

* PHRAS-1304_AUTO-COMPLETION_MASTER
fix test on media rotation 600*400 -> 400*599 !!!

* PHRAS-1304_AUTO-COMPLETION_MASTER
restore facebook sdk to 4.0.1 due to mistake

* PHRAS-1304_AUTO-COMPLETION_MASTER
deleted unwanted file
2017-02-06 18:26:56 +01:00
Thibaud Fabre
b5bbb1851f Merge branch 4.0 2016-12-05 15:26:48 +01:00
Thibaud Fabre
2aacbb789e Fix term indexer creation 2016-10-19 14:01:43 +02:00
Thibaud Fabre
b345ef7ac7 Fix indexer dependency injection 2016-10-19 13:31:39 +02:00
Thibaud Fabre
c133b42716 Refactoring
- Split search engine service registration into sub-methods
- Remove unused properties
- Fix coding style
2016-10-19 13:31:17 +02:00
Thibaud Fabre
f5452eac1e Check for undefined key 2016-10-19 13:31:17 +02:00
Thibaud Fabre
6e88b97c55 Extract record index and term index from respective indexers 2016-10-19 13:31:16 +02:00
Jean-Yves Gaulier
06a66a916f PHRAS-1271_better-buildsubdefs
- new :  doc
 - changed : some options are mandatory
 - changed : es logging to app['monolog'] to respect not-verbose mode (no more broken progress-bar due to log on screen)
2016-10-19 11:22:11 +02:00
Thibaud Fabre
79a9411923 Code architecture
- Remove circular dependency between indexer and record indexer
- Restore interface type hints on indexer methods
2016-10-18 21:17:38 +02:00
Jean-Yves Gaulier
cf40c99ea7 PHRAS-714_thesaurus_indexation
- new : if the thesaurus has been modified since it's last indexation, it will be re-indexed as soon a record is to be indexed.
 - nb : after thesaurus modification, the databox MUST be manually re-indexed (which will reindex the thesaurus before)
 - todo : remove useless dialog boxes from thesaurus app (... "this term was..."  "...add with/out reindexing ?..."). To be done in the future app using future routes.
2016-10-12 18:42:16 +02:00
Jean-Yves Gaulier
b49a5ceda5 PHRAS-1168_Backport-PHRAS-1094-to-4.0
(update php-es lib)
#time 30m
2016-06-16 17:57:04 +02:00
Benoît Burnichon
cd80e9e1ed Remove unused appbox parameter 2016-03-15 19:11:56 +01:00
Benoît Burnichon
51023c5533 bump copyright year 2016-01-05 13:38:14 +01:00
Benoît Burnichon
8d4e6a5f3f Remove debug messages when not in debug mode 2015-12-22 17:44:35 +01:00
Benoît Burnichon
663dbef92e Add Facet labels 2015-11-23 16:17:58 +01:00
Benoît Burnichon
88427efa4c Fixup DI of QueryCompiler.
As compiler was not using structure on its own, extract it and replace instantiation by a factory.
2015-11-16 13:08:32 +01:00
Mathieu Darse
3b2d6b05a0 Pass structure to query visitor & enable range generation for regular fields (equal expression) 2015-11-13 20:41:19 +01:00
Jean-Yves Gaulier
db33e2c4fd PHRAS-749 #time 2h
disable-highlight
2015-10-06 14:41:29 +02:00
Mathieu Darse
929705f13e Structure -> GlobalStructure + LimitedStructure
Rename Structure to GlobalStructure.
Add new LimitedStructure to wrap another Structure and transparently restrict
access on private fields.
2015-07-10 12:30:13 +02:00
Benoît Burnichon
6e651f3b07 Replace app['phraseanet.appbox']->get_databox using trait 2015-07-02 12:27:26 +02:00
Benoît Burnichon
dbb33c85b9 Rename GlobalElasticOptions ElasticsearchOptions 2015-07-02 12:27:25 +02:00
Benoît Burnichon
28fee99fc4 Add Form for ElasticSearch Configuration.
Also changed type from array to Options object.
2015-06-29 17:10:32 +02:00
Mathieu Darse
ea8eb4dc07 Fix another IN search regression from 40e2df6c3f
Move getIndexFieldName() from `RecordHelper` to `Field`.
Also remove an unused injected dependency
2015-06-15 19:18:06 +02:00
Benoît Burnichon
d0dee9ea2b Delay indexer instantiation, indexer subscriber can be called synchronously or asynchronously 2015-06-05 14:37:50 +02:00
Mathieu Darse
52709fb5af More semantic Structure factory 2015-06-03 19:45:48 +02:00
Mathieu Darse
421684757a Refactor merged field structure
Here is the new model:

+-----------------------------+
|          Structure          |
+-----------------------------+
| +createFromDataboxes()      |
| getAllFields()              |
| getUnrestrictedFields()     |
| getPrivateFields()          |
| getFacetsFields()           |
| getThesaurusEnabledFields() |
| getDateFields()             |
|- - - - - - - - - - - - - - -|
| add()                       |
| get()                       |
| typeOf()                    |
| isPrivate()                 |
+-------+-+-+-----------------+
        | | |          +---------------------+
        | | +--------> |        Field        |
        | |            +---------------------+
        | |            | getName()           |
        | |            | getType()           |
        | |            | isXXX()             |
        | |            | getThesaurusRoots() |
        | |            +---------------------+
        | |
        | |            +-------+
        | +----------> | Field |
        |              +-------+
        |
        |              +-------+
        +------------> | Field |
                       +-------+

It was driven by the following use cases:
- Get list of facets (only searchable fields)
- Get list of fields with concept inference
- Get list of all fields
    - Splitted in private / public fields (to define mapping)
- Get all date fields
- Get field type
    - To apply sanitization rules
    - To define mapping
- Check if concept inference enabled
- Check if the field is searchable
- Check if the field is a facet
- Check if the field is private
- Dereference field from label (still to be done)

(The last two UCs are new)

Also removed old code from legacy search engines.

[#PHRAS-500]
2015-06-03 19:45:48 +02:00
Mathieu Darse
feb7fd057e Add some indexer logging 2015-04-29 20:43:20 +02:00
Mathieu Darse
30391e89b8 Add logging capabilities to thesaurus 2015-04-29 19:08:42 +02:00
Mathieu Darse
4785fbc8ed Fix indexing issue with date fields
- Date and number types sanitization
- Remove `RecordIndexer` dependency on `ElasticSearchEngine`
- Move some sanitization from `RecordIndexer` to `RecordHelper`
2015-04-15 18:28:12 +02:00
Jean-Yves Gaulier
56fa78e0ef wip : removed Sphinx and Phrasea search engines 2015-04-08 18:37:11 +02:00
Mathieu Darse
99e62f7530 QueryParser -> QueryCompiler 2015-03-30 16:17:40 +02:00
Mathieu Darse
679106d226 Fix typos and check-in Escaper 2015-03-19 16:41:53 +01:00
Mathieu Darse
b1a03ae566 Smart facet value escaping 2015-03-19 13:42:04 +01:00
Mathieu Darse
e4ee7fc7d7 Fix broken phrasea engine (introduced in #1282) 2015-03-16 16:30:50 +01:00
Benoît Burnichon
118bb2f03c Some fixup for ES instance 2015-03-11 15:22:20 +01:00
Nicolas Le Goff
ed3ffb59a6 Merge remote-tracking branch 'upstream/master' into 20150306-elastic-indexer
Conflicts:
	bin/console
	bower.json
	composer.json
	composer.lock
	lib/Alchemy/Phrasea/Application.php
	lib/Alchemy/Phrasea/Border/Manager.php
	lib/Alchemy/Phrasea/Controller/Api/V1.php
	lib/Alchemy/Phrasea/Core/PhraseaEvents.php
	lib/Alchemy/Phrasea/SearchEngine/SearchEngineOptions.php
	lib/classes/caption/field.php
	lib/classes/record/Interface.php
	templates/web/prod/index.html.twig
	www/skins/prod/000000/prodcolor.css
2015-03-10 14:36:31 +01:00
Mathieu Darse
e889d19b7d Filter databoxes earlier (in Indexer) and databox id error handling 2015-03-10 11:52:30 +01:00
Mathieu Darse
a84602f896 Move concept injection into query parser 2015-02-25 16:12:58 +01:00