hazza/Phraseanet
1
0
Fork 0
mirror of https://github.com/alchemy-fr/Phraseanet.git synced 2025-10-12 20:43:25 +00:00
Code Issues Packages Projects Releases Wiki Activity
15,668 Commits 268 Branches 107 Tags
00979f2d6d256e8b8ff580090acb5ba80f498ebd
Commit Graph

84 Commits

Author SHA1 Message Date
jygaulier
cfd385498d - add : cache on es "tags" 
- removed : variable globalstructure construction (WITH_FIELDS, WITH_FLAGS...) (was unused)
- cleanup (stopwatches etc)
 2020-11-05 19:54:14 +01:00
jygaulier
5ae105d9c2 - fix : error during populate 
- add : clear structure cache when structure (field or sb) is changed
 2020-11-05 11:27:39 +01:00
jygaulier
4d961e2dbe - fix : added cache to es "globalstructure" construction 2020-11-04 11:35:36 +01:00
jygaulier
e36b0785ee - add some simple cache 
- disable check of conf/searchengine/type
- faster construction of concept-paths
- pass strucure to es as factory (useless, to be reverted)
- add stopwatch (log into file, disabled for now)
 2020-10-30 11:00:17 +01:00
Jean-Yves Gaulier
8ffeb58289 PHRAS-3001_es-geopoint-sources_4.1 
WIP
- fix : gps position indexed by es now comes from the metadata fields defined in conf; (not from exif anymore).
- todo/wip : decide conf place, since today we only have an array of geo providers
- todo/wip : does exif apply as fallback if nothing in conf ? or if metadata not set in record ?
 2020-03-29 20:11:03 +02:00
Jean-Yves Gaulier
20839f1dd7 return facets following the order defined in conf. 2020-02-17 20:17:08 +01:00
Jean-Yves Gaulier
53f17e1897 save modifications in conf, and also write back to databoxes 2020-02-17 20:17:08 +01:00
aynsix
df85aa8827 PHRAS-1797 porting searchengine:index to 4.1 (#2454) 
* portage commande searchengine:index vers 4.1

* FIX Indexer

* Update DataboxFetcherFactory.php
 2018-01-30 18:50:57 +01:00
jygaulier
a70bf2fc70 PHRAS-1304_AUTO-COMPLETION_MASTER (#2061) 
* PHRAS-1304_AUTO-COMPLETION_MASTER
ported from 4.0

* PHRAS-1304_AUTO-COMPLETION_MASTER
fix

* PHRAS-1304_AUTO-COMPLETION_MASTER
fix

* PHRAS-1304_AUTO-COMPLETION_MASTER
bump php version to 5.5.31 (5.5.21 is obsolete in cicleci)

* PHRAS-1304_AUTO-COMPLETION_MASTER
bump php version to 5.5.31 : php.ini moved in circelci

* PHRAS-1304_AUTO-COMPLETION_MASTER
add zmq & date to php for circleci

* PHRAS-1304_AUTO-COMPLETION_MASTER
add zmq

* PHRAS-1304_AUTO-COMPLETION_MASTER
bump amqp

* PHRAS-1304_AUTO-COMPLETION_MASTER
downgrade amqp to 1.2 to test compilation against old librabbit 0.4 (ubuntu)

* PHRAS-1304_AUTO-COMPLETION_MASTER
add amqp.so to php.ini, (re)bump amqp to 1.6

* PHRAS-1304_AUTO-COMPLETION_MASTER
build rabittmq from git

* PHRAS-1304_AUTO-COMPLETION_MASTER
build rabittmq from git again

* PHRAS-1304_AUTO-COMPLETION_MASTER
build rabittmq from git again and again

* PHRAS-1304_AUTO-COMPLETION_MASTER
fix test on media rotation 600*400 -> 400*599 !!!

* PHRAS-1304_AUTO-COMPLETION_MASTER
restore facebook sdk to 4.0.1 due to mistake

* PHRAS-1304_AUTO-COMPLETION_MASTER
deleted unwanted file
 2017-02-06 18:26:56 +01:00
Thibaud Fabre
b5bbb1851f Merge branch 4.0 2016-12-05 15:26:48 +01:00
Thibaud Fabre
2aacbb789e Fix term indexer creation 2016-10-19 14:01:43 +02:00
Thibaud Fabre
b345ef7ac7 Fix indexer dependency injection 2016-10-19 13:31:39 +02:00
Thibaud Fabre
c133b42716 Refactoring 
- Split search engine service registration into sub-methods
- Remove unused properties
- Fix coding style
 2016-10-19 13:31:17 +02:00
Thibaud Fabre
f5452eac1e Check for undefined key 2016-10-19 13:31:17 +02:00
Thibaud Fabre
6e88b97c55 Extract record index and term index from respective indexers 2016-10-19 13:31:16 +02:00
Jean-Yves Gaulier
06a66a916f PHRAS-1271_better-buildsubdefs 
- new :  doc
 - changed : some options are mandatory
 - changed : es logging to app['monolog'] to respect not-verbose mode (no more broken progress-bar due to log on screen)
 2016-10-19 11:22:11 +02:00
Thibaud Fabre
79a9411923 Code architecture 
- Remove circular dependency between indexer and record indexer
- Restore interface type hints on indexer methods
 2016-10-18 21:17:38 +02:00
Jean-Yves Gaulier
cf40c99ea7 PHRAS-714_thesaurus_indexation 
- new : if the thesaurus has been modified since it's last indexation, it will be re-indexed as soon a record is to be indexed.
 - nb : after thesaurus modification, the databox MUST be manually re-indexed (which will reindex the thesaurus before)
 - todo : remove useless dialog boxes from thesaurus app (... "this term was..."  "...add with/out reindexing ?..."). To be done in the future app using future routes.
 2016-10-12 18:42:16 +02:00
Jean-Yves Gaulier
b49a5ceda5 PHRAS-1168_Backport-PHRAS-1094-to-4.0 
(update php-es lib)
#time 30m
 2016-06-16 17:57:04 +02:00
Benoît Burnichon
cd80e9e1ed Remove unused appbox parameter 2016-03-15 19:11:56 +01:00
Benoît Burnichon
51023c5533 bump copyright year 2016-01-05 13:38:14 +01:00
Benoît Burnichon
8d4e6a5f3f Remove debug messages when not in debug mode 2015-12-22 17:44:35 +01:00
Benoît Burnichon
663dbef92e Add Facet labels 2015-11-23 16:17:58 +01:00
Benoît Burnichon
88427efa4c Fixup DI of QueryCompiler. 
As compiler was not using structure on its own, extract it and replace instantiation by a factory.
 2015-11-16 13:08:32 +01:00
Mathieu Darse
3b2d6b05a0 Pass structure to query visitor & enable range generation for regular fields (equal expression) 2015-11-13 20:41:19 +01:00
Jean-Yves Gaulier
db33e2c4fd PHRAS-749 #time 2h 
disable-highlight
 2015-10-06 14:41:29 +02:00
Mathieu Darse
929705f13e Structure -> GlobalStructure + LimitedStructure 
Rename Structure to GlobalStructure.
Add new LimitedStructure to wrap another Structure and transparently restrict
access on private fields.
 2015-07-10 12:30:13 +02:00
Benoît Burnichon
6e651f3b07 Replace app['phraseanet.appbox']->get_databox using trait 2015-07-02 12:27:26 +02:00
Benoît Burnichon
dbb33c85b9 Rename GlobalElasticOptions ElasticsearchOptions 2015-07-02 12:27:25 +02:00
Benoît Burnichon
28fee99fc4 Add Form for ElasticSearch Configuration. 
Also changed type from array to Options object.
 2015-06-29 17:10:32 +02:00
Mathieu Darse
ea8eb4dc07 Fix another IN search regression from 40e2df6c3f 
Move getIndexFieldName() from `RecordHelper` to `Field`.
Also remove an unused injected dependency
 2015-06-15 19:18:06 +02:00
Benoît Burnichon
d0dee9ea2b Delay indexer instantiation, indexer subscriber can be called synchronously or asynchronously 2015-06-05 14:37:50 +02:00
Mathieu Darse
52709fb5af More semantic Structure factory 2015-06-03 19:45:48 +02:00
Mathieu Darse
421684757a Refactor merged field structure 
Here is the new model:

+-----------------------------+
|          Structure          |
+-----------------------------+
| +createFromDataboxes()      |
| getAllFields()              |
| getUnrestrictedFields()     |
| getPrivateFields()          |
| getFacetsFields()           |
| getThesaurusEnabledFields() |
| getDateFields()             |
|- - - - - - - - - - - - - - -|
| add()                       |
| get()                       |
| typeOf()                    |
| isPrivate()                 |
+-------+-+-+-----------------+
        | | |          +---------------------+
        | | +--------> |        Field        |
        | |            +---------------------+
        | |            | getName()           |
        | |            | getType()           |
        | |            | isXXX()             |
        | |            | getThesaurusRoots() |
        | |            +---------------------+
        | |
        | |            +-------+
        | +----------> | Field |
        |              +-------+
        |
        |              +-------+
        +------------> | Field |
                       +-------+

It was driven by the following use cases:
- Get list of facets (only searchable fields)
- Get list of fields with concept inference
- Get list of all fields
    - Splitted in private / public fields (to define mapping)
- Get all date fields
- Get field type
    - To apply sanitization rules
    - To define mapping
- Check if concept inference enabled
- Check if the field is searchable
- Check if the field is a facet
- Check if the field is private
- Dereference field from label (still to be done)

(The last two UCs are new)

Also removed old code from legacy search engines.

[#PHRAS-500]
 2015-06-03 19:45:48 +02:00
Mathieu Darse
feb7fd057e Add some indexer logging 2015-04-29 20:43:20 +02:00
Mathieu Darse
30391e89b8 Add logging capabilities to thesaurus 2015-04-29 19:08:42 +02:00
Mathieu Darse
4785fbc8ed Fix indexing issue with date fields 
- Date and number types sanitization
- Remove `RecordIndexer` dependency on `ElasticSearchEngine`
- Move some sanitization from `RecordIndexer` to `RecordHelper`
 2015-04-15 18:28:12 +02:00
Jean-Yves Gaulier
56fa78e0ef wip : removed Sphinx and Phrasea search engines 2015-04-08 18:37:11 +02:00
Mathieu Darse
99e62f7530 QueryParser -> QueryCompiler 2015-03-30 16:17:40 +02:00
Mathieu Darse
679106d226 Fix typos and check-in Escaper 2015-03-19 16:41:53 +01:00
Mathieu Darse
b1a03ae566 Smart facet value escaping 2015-03-19 13:42:04 +01:00
Mathieu Darse
e4ee7fc7d7 Fix broken phrasea engine (introduced in #1282) 2015-03-16 16:30:50 +01:00
Benoît Burnichon
118bb2f03c Some fixup for ES instance 2015-03-11 15:22:20 +01:00
Nicolas Le Goff
ed3ffb59a6 Merge remote-tracking branch 'upstream/master' into 20150306-elastic-indexer 
Conflicts:
	bin/console
	bower.json
	composer.json
	composer.lock
	lib/Alchemy/Phrasea/Application.php
	lib/Alchemy/Phrasea/Border/Manager.php
	lib/Alchemy/Phrasea/Controller/Api/V1.php
	lib/Alchemy/Phrasea/Core/PhraseaEvents.php
	lib/Alchemy/Phrasea/SearchEngine/SearchEngineOptions.php
	lib/classes/caption/field.php
	lib/classes/record/Interface.php
	templates/web/prod/index.html.twig
	www/skins/prod/000000/prodcolor.css
 2015-03-10 14:36:31 +01:00
Mathieu Darse
e889d19b7d Filter databoxes earlier (in Indexer) and databox id error handling 2015-03-10 11:52:30 +01:00
Mathieu Darse
a84602f896 Move concept injection into query parser 2015-02-25 16:12:58 +01:00
Nicolas Le Goff
7299928840 Load service provider if app is setup 2015-02-24 15:25:43 +01:00
Nicolas Le Goff
949bf06cac Merge branch '3.8' 
Conflicts:
	CHANGELOG.md
	bin/console
	bin/developer
	bin/setup
	bower.json
	composer.json
	composer.lock
	features/bootstrap/FeatureContext.php
	features/bootstrap/GuiContext.php
	lib/Alchemy/Phrasea/Authentication/Token/TokenValidator.php
	lib/Alchemy/Phrasea/Command/BuildMissingSubdefs.php
	lib/Alchemy/Phrasea/Command/CreateCollection.php
	lib/Alchemy/Phrasea/Command/Developer/JavascriptBuilder.php
	lib/Alchemy/Phrasea/Controller/Admin/Collection.php
	lib/Alchemy/Phrasea/Controller/Admin/Databoxes.php
	lib/Alchemy/Phrasea/Controller/Admin/TaskManager.php
	lib/Alchemy/Phrasea/Controller/Api/V1.php
	lib/Alchemy/Phrasea/Controller/Client/Baskets.php
	lib/Alchemy/Phrasea/Controller/Client/Root.php
	lib/Alchemy/Phrasea/Controller/Prod/Basket.php
	lib/Alchemy/Phrasea/Controller/Prod/Export.php
	lib/Alchemy/Phrasea/Controller/Prod/Property.php
	lib/Alchemy/Phrasea/Controller/Prod/Records.php
	lib/Alchemy/Phrasea/Controller/Prod/Tools.php
	lib/Alchemy/Phrasea/Controller/Prod/Upload.php
	lib/Alchemy/Phrasea/Controller/Root/Login.php
	lib/Alchemy/Phrasea/Controller/Thesaurus/Thesaurus.php
	lib/Alchemy/Phrasea/Core/Event/ApiLoadEndEvent.php
	lib/Alchemy/Phrasea/Core/Event/ApiLoadStartEvent.php
	lib/Alchemy/Phrasea/Core/Provider/TaskManagerServiceProvider.php
	lib/Alchemy/Phrasea/Core/Version.php
	lib/Alchemy/Phrasea/Exception/XMLParseErrorException.php
	lib/Alchemy/Phrasea/Helper/DatabaseHelper.php
	lib/Alchemy/Phrasea/Helper/User/Edit.php
	lib/Alchemy/Phrasea/SearchEngine/Phrasea/PhraseaEngine.php
	lib/Alchemy/Phrasea/SearchEngine/SearchEngineOptions.php
	lib/Doctrine/Entities/AuthFailure.php
	lib/Doctrine/Entities/Basket.php
	lib/Doctrine/Entities/BasketElement.php
	lib/Doctrine/Entities/LazaretAttribute.php
	lib/Doctrine/Entities/LazaretCheck.php
	lib/Doctrine/Entities/LazaretFile.php
	lib/Doctrine/Entities/LazaretSession.php
	lib/Doctrine/Entities/Session.php
	lib/Doctrine/Entities/SessionModule.php
	lib/Doctrine/Entities/StoryWZ.php
	lib/Doctrine/Entities/UsrList.php
	lib/Doctrine/Entities/UsrListEntry.php
	lib/Doctrine/Entities/UsrListOwner.php
	lib/Doctrine/Entities/ValidationData.php
	lib/Doctrine/Entities/ValidationParticipant.php
	lib/Doctrine/Entities/ValidationSession.php
	lib/Doctrine/Logger/MonologSQLLogger.php
	lib/Doctrine/Repositories/BasketRepository.php
	lib/Doctrine/Repositories/ValidationParticipantRepository.php
	lib/Doctrine/Types/Binary.php
	lib/Doctrine/Types/Blob.php
	lib/Doctrine/Types/Enum.php
	lib/Doctrine/Types/LongBlob.php
	lib/Doctrine/Types/VarBinary.php
	lib/classes/API/OAuth2/Account.php
	lib/classes/API/OAuth2/Application.php
	lib/classes/API/OAuth2/Application/OfficePlugin.php
	lib/classes/API/OAuth2/AuthCode.php
	lib/classes/API/OAuth2/RefreshToken.php
	lib/classes/API/OAuth2/Token.php
	lib/classes/API/V1/Abstract.php
	lib/classes/API/V1/Interface.php
	lib/classes/API/V1/adapter.php
	lib/classes/API/V1/exception/abstract.php
	lib/classes/API/V1/exception/badrequest.php
	lib/classes/API/V1/exception/forbidden.php
	lib/classes/API/V1/exception/internalservererror.php
	lib/classes/API/V1/exception/maintenance.php
	lib/classes/API/V1/exception/methodnotallowed.php
	lib/classes/API/V1/exception/notfound.php
	lib/classes/API/V1/exception/unauthorized.php
	lib/classes/API/V1/result.php
	lib/classes/Exception/Feed/EntryNotFound.php
	lib/classes/Exception/Feed/ItemNotFound.php
	lib/classes/Exception/Feed/PublisherNotFound.php
	lib/classes/Feed/Abstract.php
	lib/classes/Feed/Adapter.php
	lib/classes/Feed/Aggregate.php
	lib/classes/Feed/Collection.php
	lib/classes/Feed/CollectionInterface.php
	lib/classes/Feed/Entry/Adapter.php
	lib/classes/Feed/Entry/Collection.php
	lib/classes/Feed/Entry/CollectionInterface.php
	lib/classes/Feed/Entry/Interface.php
	lib/classes/Feed/Entry/Item.php
	lib/classes/Feed/Entry/ItemInterface.php
	lib/classes/Feed/Interface.php
	lib/classes/Feed/Link.php
	lib/classes/Feed/LinkInterface.php
	lib/classes/Feed/Publisher/Adapter.php
	lib/classes/Feed/Publisher/Interface.php
	lib/classes/Feed/Token.php
	lib/classes/Feed/TokenAggregate.php
	lib/classes/Feed/XML/Abstract.php
	lib/classes/Feed/XML/Atom.php
	lib/classes/Feed/XML/Cooliris.php
	lib/classes/Feed/XML/Interface.php
	lib/classes/Feed/XML/RSS.php
	lib/classes/Feed/XML/RSS/Image.php
	lib/classes/Feed/XML/RSS/ImageInterface.php
	lib/classes/User/Adapter.php
	lib/classes/User/Interface.php
	lib/classes/appbox/register.php
	lib/classes/connection.php
	lib/classes/connection/abstract.php
	lib/classes/connection/interface.php
	lib/classes/connection/pdo.php
	lib/classes/connection/pdoStatementDebugger.php
	lib/classes/deprecated/countries.php
	lib/classes/deprecated/inscript.api.php
	lib/classes/eventsmanager/event/test.php
	lib/classes/ftpclient.php
	lib/classes/http/request.php
	lib/classes/media/subdef.php
	lib/classes/module/console/schedulerStart.php
	lib/classes/module/console/schedulerState.php
	lib/classes/module/console/schedulerStop.php
	lib/classes/module/console/taskState.php
	lib/classes/module/console/tasklist.php
	lib/classes/module/console/taskrun.php
	lib/classes/patch/320alpha4b.php
	lib/classes/patch/3715alpha1a.php
	lib/classes/patch/379alpha1a.php
	lib/classes/patch/380alpha10a.php
	lib/classes/patch/380alpha11a.php
	lib/classes/patch/380alpha13a.php
	lib/classes/patch/380alpha14a.php
	lib/classes/patch/380alpha15a.php
	lib/classes/patch/380alpha16a.php
	lib/classes/patch/380alpha17a.php
	lib/classes/patch/380alpha18a.php
	lib/classes/patch/380alpha3a.php
	lib/classes/patch/380alpha4a.php
	lib/classes/patch/380alpha6a.php
	lib/classes/patch/380alpha8a.php
	lib/classes/patch/380alpha9a.php
	lib/classes/patch/381alpha1b.php
	lib/classes/patch/381alpha2a.php
	lib/classes/patch/381alpha3a.php
	lib/classes/patch/381alpha4a.php
	lib/classes/patch/383alpha1a.php
	lib/classes/patch/383alpha2a.php
	lib/classes/patch/383alpha3a.php
	lib/classes/patch/383alpha4a.php
	lib/classes/record/adapter.php
	lib/classes/record/preview.php
	lib/classes/recordutils.php
	lib/classes/recordutils/audio.php
	lib/classes/recordutils/document.php
	lib/classes/recordutils/map.php
	lib/classes/recordutils/video.php
	lib/classes/registry.php
	lib/classes/registryInterface.php
	lib/classes/set/order.php
	lib/classes/system/url.php
	lib/classes/task/Scheduler.php
	lib/classes/task/appboxAbstract.php
	lib/classes/task/databoxAbstract.php
	lib/classes/task/manager.php
	lib/classes/task/period/RecordMover.php
	lib/classes/task/period/apibridge.php
	lib/classes/task/period/apiwebhooks.php
	lib/classes/task/period/archive.php
	lib/classes/task/period/cindexer.php
	lib/classes/task/period/emptyColl.php
	lib/classes/task/period/ftp.php
	lib/classes/task/period/ftpPull.php
	lib/classes/task/period/subdef.php
	lib/classes/task/period/test.php
	lib/classes/task/period/writemeta.php
	lib/conf.d/PhraseaFixture/AbstractWZ.php
	lib/conf.d/PhraseaFixture/Basket/LoadFiveBaskets.php
	lib/conf.d/PhraseaFixture/Basket/LoadOneBasket.php
	lib/conf.d/PhraseaFixture/Basket/LoadOneBasketEnv.php
	lib/conf.d/PhraseaFixture/Lazaret/LoadOneFile.php
	lib/conf.d/PhraseaFixture/Story/LoadOneStory.php
	lib/conf.d/PhraseaFixture/UsrLists/ListAbstract.php
	lib/conf.d/PhraseaFixture/UsrLists/UsrList.php
	lib/conf.d/PhraseaFixture/UsrLists/UsrListEntry.php
	lib/conf.d/PhraseaFixture/UsrLists/UsrListOwner.php
	lib/conf.d/PhraseaFixture/ValidationParticipant/LoadOneParticipant.php
	lib/conf.d/PhraseaFixture/ValidationParticipant/LoadParticipantWithSession.php
	lib/conf.d/PhraseaFixture/ValidationSession/LoadOneValidationSession.php
	templates/web/admin/collection/collection.html.twig
	templates/web/common/dialog_export.html.twig
	templates/web/common/menubar.html.twig
	templates/web/prod/actions/Tools/index.html.twig
	templates/web/prod/index.html.twig
	templates/web/prod/upload/upload-flash.html.twig
	templates/web/prod/upload/upload.html.twig
	templates/web/report/report_layout_child.html.twig
	templates/web/setup/step2.html.twig
	templates/web/thesaurus/new-synonym-dialog.html.twig
	templates/web/thesaurus/properties.html.twig
	templates/web/thesaurus/search.html.twig
	tests/Alchemy/Tests/Phrasea/Application/ApiAbstract.php
	tests/Alchemy/Tests/Phrasea/Cache/FactoryTest.php
	tests/Alchemy/Tests/Phrasea/Controller/Admin/AdminCollectionTest.php
	tests/Alchemy/Tests/Phrasea/Controller/Client/RootTest.php
 2015-02-05 18:38:49 +01:00
Mathieu Darse
a0dad0e96e WIP on reindexing events handling 2015-02-04 19:53:35 +01:00
Mathieu Darse
dc2c9f8c7f Refactor thesaurus query build 
- Look for text nodes and infer the concepts behind term using thesaurus
- Use value objects for thesaurus terms and concepts
- Pass a QueryContext holding allowed fields and locales informations when building the Elasticsearch query
- Change type hinting and name of query building method on nodes
- Remove unused method Node#isFullTextOnly()
- Move getFieldsStructure from RecordIndexer to RecordHelper for reusing field structure in SearchEngine
 2015-01-20 15:05:15 +01:00