diff --git a/dspace/docs/application.html b/dspace/docs/application.html new file mode 100644 index 0000000000..50cee53e6b --- /dev/null +++ b/dspace/docs/application.html @@ -0,0 +1,1170 @@ + + + + DSpace System +Documentation: Application Layer + + + + +

DSpace +System Documentation: +Application Layer

+

Back to contents
+Back +to architecture overview

+

Web User Interface

+

The DSpace Web UI is the +largest and most-used component in the application layer. Built on Java +Servlet and JavaServer Page technology, it allows end-users to access +DSpace over the Web via their Web browsers.

+

It also features an +administration section, consisting of pages intended for use by central +administrators. Presently, this part of the Web UI is not particularly +sophisticated; users of the administration section need to know what +they are doing! Selected parts of this may also be used by collection +[FIXME: administrators or editors?]

+

Web UI Files

+

The Web UI-related files are +located in a variety of directories in the DSpace source tree. Note +that as of DSpace version 1.2, the deployment mechanism has changed; +the build process creates easy-to-deploy Web application archives (.war +files).

+ + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + +
Locations of Web UI +Source Files
LocationDescription
org.dspace.app.webuiWeb UI source files
org.dspace.app.webui.filterServlet Filters (Servlet +2.3 spec)
org.dspace.app.webui.jsptagCustom JSP tag class +files
org.dspace.app.webui.servletServlets for main Web UI +(controllers)
org.dspace.app.webui.servlet.adminServlets that comprise +the administration part of the Web UI
org.dspace.app.webui.utilMiscellaneous classes +used by the servlets and filters
[dspace-source]/jspThe JSP files
[dspace-source]/jsp/localThis is where you can +place customized versions of JSPs -- see the configuration +section
[dspace-source]/jsp/WEB-INF/dspace-tags.tldCustom DSpace JSP tag +descriptor
[dspace-source]/etc/dspace-web.xmlThe Web application +deployment descriptor. Before including in the .war +file, the text @@dspace.dir@@ +will be replaced with the DSpace installation directory (referred to as + [dspace] +elsewhere in this system documentation). This allows the Web +application to pick up the DSpace configuration and environment.
+

The Build Process

+

The DSpace build process +constructs a Web application archive, which is placed in [dspace-source]/build/dspace.war. +The build_wars +Ant target does the work. The process works as follows:

+ +

The contents of dspace.war +are:

+ +

Note that this does mean there +are multiple copies of the compiled DSpace code and third-party +libraries in the system, so care must be taken to ensure that they are +all in sync. (The storage overhead is a few megabytes, totally +insignificant these days.) In general, when you change any DSpace code +or JSP, it's best to do a complete update of both the installation ([dspace]), +and to rebuild and redeploy the Web UI and OAI .war +files, by running this in [dspace-source]:

+
ant -D[dspace]/config/dspace.cfg update
+

and then following the +instructions that command writes to the console.

+

Servlets and JSPs

+

The Web UI is loosely based +around the MVC (model, view, controller) model. The content management +API corresponds to the model, the Java Servlets are the controllers, +and the JSPs are the views. Interactions take the following basic form:

+
    +
  1. An HTTP request is received +from a browser
    2. +
  2. The appropriate servlet is +invoked, and processes the request by invoking the DSpace business +logic layer public API
    3. +
  3. Depending on the outcome of +the processing, the servlet invokes the appropriate JSP
    4. +
  4. The JSP is processed and +sent to the browser
    5. +
+

The reasons for this approach +are:

+ +

The org.dspace.app.webui.servlet.LoadDSpaceConfig +servlet is always loaded first. This is a very simple servlet that +checks the dspace-config +context parameter from the DSpace deployment descriptor, and uses it to +locate dspace.cfg. +It also loads up the Log4j configuration. It's important that this +servlet is loaded first, since if another servlet is loaded up, it will +cause the system to try and load DSpace and Log4j configurations, +neither of which would be found.

+

All DSpace servlets are +subclasses of the DSpaceServlet +class. The DSpaceServlet +class handles some basic operations such as creating a DSpace Context +object (opening a database connection etc.), authentication and error +handling. Instead of overriding the doGet +and doPost +methods as one normally would for a servlet, DSpace servlets implement doDSGet +or doDSPost +which have an extra context parameter, and allow the servlet to throw +various exceptions that can be handled in a standard way.

+

The DSpace servlet processes +the contents of the HTTP request. This might involve retrieving the +results of a search with a query term, accessing the current user's +eperson record, or updating a submission in progress. According to the +results of this processing, the servlet must decide which JSP should be +displayed. The servlet then fills out the appropriate attributes in the +HttpRequest +object that represents the HTTP request being processed. This is done +by invoking the setAttribute +method of the javax.servlet.http.HttpServletRequest +object that is passed into the servlet from Tomcat. The servlet then +forwards control of the request to the appropriate JSP using the JSPManager.showJSP +method.

+

The JSPManager.showJSP +method uses the standard Java servlet forwarding mechanism is then used +to forward the HTTP request to the JSP. The JSP is processed by Tomcat +and the results sent back to the user's browser.

+

There is an exception to this +servlet/JSP style: index.jsp, +the 'home page', receives the HTTP request directly from Tomcat without +a servlet being invoked first. This is because in the servlet 2.3 +specification, there is no way to map a servlet to handle only requests +made to '/'; +such a mapping results in every request being directed to that servlet. +By default, Tomcat forwards requests to '/' +to index.jsp. +To try and make things as clean as possible, index.jsp +contains some simple code that would normally go in a servlet, and then +forwards to home.jsp +using the JSPManager.showJSP +method. This means localized versions of the 'home page' can be created +by placing a customized home.jsp +in [dspace-source]/jsp/local, +in the same manner as other JSPs.

+

[dspace-source]/jsp/dspace-admin/index.jsp, +the administration UI index page, is invoked directly by Tomcat and not +through a servlet for similar reasons.

+

At the top of each JSP file, +right after the license and copyright header, is documented the +appropriate attributes that a servlet must fill out prior to forwarding +to that JSP. No validation is performed; if the servlet does not fill +out the necessary attributes, it is likely that an internal server +error will occur.

+

Many JSPs containing forms will +include hidden parameters that tell the servlets which form has been +filled out. The submission UI servlet (SubmitServlet +is a prime example of a servlet that deals with the input from many +different JSPs. The step +hidden parameter is used to inform the servlet which form has been +filled out (which step of submission the user has just completed.)

+

Below is a detailed, scary +diagram depicting the flow of control during the whole process of +processing and responding to an HTTP request. More information about +the authentication mechanism is mostly described in the +configuration section.

+

Web UI Control Flow

+

Flow of +Control During HTTP Request Processing

+

Custom JSP Tags

+

The DSpace JSPs all use some +custom tags defined in /dspace/jsp/WEB-INF/dspace-tags.tld, +and the corresponding Java classes reside in org.dspace.app.webui.jsptag. +The tags are listed below. The dspace-tags.tld +file contains detailed comments about how to use the tags, so that +information is not repeated here.

+
+
layout
+
+

Just about every JSP uses +this tag. It produces the standard HTML header and <BODY>tag. +Thus the content of each JSP is nested inside a <dspace:layout> +tag. The (XML-style)attributes of this tag are slightly +complicated--see dspace-tags.tld. +The JSPs in the source code bundle also provide plenty of examples.

+
+
sidebar
+
+

Can only be used inside a layout +tag, and can only be used once per JSP. The content between the start +and end sidebar +tags is rendered in a column on the right-hand side of the HTML page. +The contents can contain further JSP tags and Java 'scriptlets'.

+
+
date
+
+

Displays the date +represented by an org.dspace.content.DCDate +object. Just the one representation of date is rendered currently, but +this could use the user's browser preferences to display a localized +date in the future.

+
+
include
+
+

Obsolete, simple tag, +similar to jsp:include. +In versions prior to DSpace 1.2, this tag would use the locally +modified version of a JSP if one was installed in jsp/local. As of 1.2, +the build process now performs this function, however this tag is left +in for backwards compatibility.

+
+
item
+
+

Displays an item record, +including Dublin Core metadata and links to the bitstreams within it. +Note that the displaying of the bitstream links is simplistic, and does +not take into account any of the bundling structure. This is because +DSpace does not have a fully-fledged dissemination architectural piece +yet.

+

Displaying an item record +is done by a tag rather than a JSP for two reasons: Firstly, it happens +in several places (when verifying an item record during submission or +workflow review, as well as during standard item accesses), and +secondly, displaying the item turns out to be mostly code-work rather +than HTML anyway. Of course, the disadvantage of doing it this way is +that it is slightly harder to customize exactly what is displayed from +an item record; it is necessary to edit the tag code (org.dspace.app.webui.jsptag.ItemTag). +Hopefully a better solution can be found in the future.

+
+
itemlist, collectionlist, communitylist
+
+

These tags display ordered +sequences of items, collections and communities, showing minimal +information but including a link to the page containing full details. +These need to be used in HTML tables.

+
+
popup
+
+

This tag is used to render +a link to a pop-up page (typically a help page.) If Javascript is +available, the link will either open or pop to the front any existing +DSpace pop-up window. If Javascript is not available, a standard HTML +link is displayed that renders the link destination in a window named 'dspace.popup'. +In graphical browsers, this usually opens a new window or re-uses an +existing window of that name, but if a window is re-used it is not +'raised' which might confuse the user. In text browsers, following this +link will simply replace the current page with the destination of the +link. This obviously means that Javascript offers the best +functionality, but other browsers are still supported.

+
+
selecteperson
+
+

A tag which produces a +widget analogous to HTML <SELECT>, +that allows a user to select one or multiple e-people from a pop-up +list.

+
+
sfxlink
+
+

Using an item's Dublin Core +metadata DSpace can display an SFX link, if an SFX server is available. +This tag does so for a particular item if the sfx.server.url +property is defined in dspace.cfg.

+
+
+

HTML Support

+

For the most part, the DSpace +item display just gives a link that allows an end-user to download a +bitstream. However, if a bundle has a primary bitstream whose format is +of MIME type text/html, +instead a link to the HTML servlet is given.

+

So if we had an HTML document +like this:

+
contents.html
chapter1.html
chapter2.html
chapter3.html
figure1.gif
figure2.jpg
figure3.gif
figure4.jpg
figure5.gif
figure6.gif
+

The Bundle's primary bitstream +field would point to the contents.html Bitstream, which we know is HTML +(check the format MIME type) and so we know which to serve up first.

+

The HTML servlet employs a +trick to serve up HTML documents without actually modifying the HTML or +other files themselves. Say someone is looking at contents.html +from the above example, the URL in their browser will look like this:

+
https://dspace.mit.edu/html/1721.1/12345/contents.html
+

If there's an image called figure1.gif +in that HTML page, the browser will do HTTP GET on this URL:

+
https://dspace.mit.edu/html/1721.1/12345/figure1.gif
+

The HTML document servlet can +work out which item the user is looking at, and then which Bitstream in +it is called figure1.gif, +and serve up that bitstream. Similar for following links to other HTML +pages. Of course all the links and image references have to be relative +and not absolute.

+

This can cope with relative +links that refer to a deeper path, e.g.

+
<IMG SRC="images/figure1.gif">
+

Remember that in the Bitstream +table in the database we have the 'name' field, which always contains +the filename with no path (figure1.gif). +We also have the source +field, which may +contain the full pathname of the file as it appeared on the submitter's +hard drive, but this is browser- and OS-dependent, so we can't rely on +it. All we can rely on is the filename.

+

We can still work out what +images/figure1.gif is by making the HTML document servlet strip any +path that comes in from the URL, e.g.

+
https://dspace.mit.edu/html/1721.1/12345/images/figure1.gif
                                         ^^^^^^^
                                        Strip this
+

BUT all the filenames +(regardless of directory names) must be unique. For example, this +wouldn't work:

+
contents.html
chapter1.html
chapter2.html
chapter1_images/figure.gif
chapter2_images/figure.gif
+

since the HTML document servlet +wouldn't know which bitstream to serve up for:

+
https://dspace.mit.edu/html/1721.1/12345/chapter1_images/figure.gif
https://dspace.mit.edu/html/1721.1/12345/chapter2_images/figure.gif
+

since it would just have figure.gif +in the Bitstream table. Thus, the limitations are:

+ +

Thesis Blocking

+

The submission UI has an +optional feature that came about as a result of MIT Libraries policy. +If the block.theses +parameter in dspace.cfg +is true, +an extra checkbox is included in the first page of the submission UI. +This asks the user if the submission is a thesis. If the user checks +this box, the submission is halted (deleted) and an error message +displayed, explaining that DSpace should not be used to submit theses. +This feature can be turned off and on, and the message displayed (/dspace/jsp/submit/no-theses.jsp +can be localized as necessary.

+

OAI-PMH Data Provider

+

The DSpace platform supports +the Open +Archives Initiative Protocol for Metadata Harvesting +(OAI-PMH) version 2.0 as a data provider. This is accomplished using +the OAICat +framework from OCLC.

+

The DSpace build process builds +a Web application archive, [dspace-source]/build/dspace-oai.war), +in much the same way as the Web UI build process +described above. The only differences are that the JSPs are not +included, and [dspace-source]/etc/oai-web.xml +is used as the deployment descriptor. This 'webapp' is deployed to +receive and respond to OAI-PMH requests via HTTP. Note that typically +it should not +be deployed on SSL (https: +protocol). In a typical configuration, this is deployed at dspace-oai, +for example:

+
http://dspace.myu.edu/dspace-oai/request?verb=Identify
+

The 'base URL' of this DSpace +deployment would be:

+
http://dspace.myu.edu/dspace-oai/request
+

It is this URL that should be +registered with www.openarchives.org. +Note that you can easily change the 'request' +portion of the URL by editing [dspace-source]/etc/oai-web.xml +and rebuilding and deploying dspace-oai.war.

+

DSpace provides implementations +of the OAICat interfaces AbstractCatalog, +RecordFactory +and Crosswalk +that interface with the DSpace content management API and harvesting +API (in the search subsystem).

+

Only the basic oai_dc +unqualified Dublin Core metadata set is exported at present; this is +particularly easy since all items have qualified Dublin Core metadata. +When this metadata is harvested, the qualifiers are simply stripped; +for example, description.abstract +is exposed as unqualified description. +The description.provenance +field is hidden, as this contains private information about the +submitter and workflow reviewers of the item, including their e-mail +addresses. Additionally, to keep in line with OAI community practices, +values of contributor.author +are exposed as creator +values.

+

To add support for other +metadata sets is simply a matter of creating another Crosswalk +implementation, and adding it to the oaicat.properties +file described below.

+

Note that the current simple DC +implementation (org.dspace.app.oai.OAIDCCrosswalk) +does not currently strip out any invalid XML characters that may be +lying around in the data. If your database contains a DC value with, +for example, some ASCII control codes (form feed etc.) this may cause +OAI harvesters problems. This should rarely occur, however. XML +entities (such as >) +are encoded (e.g. to &gt;)

+

In addition to the +implementations of the OAICat interfaces, there are two configuration +files relevant to OAI support:

+
+
oaicat.properties
+
+

This resides as a template +in [dspace]/config/templates, +and the live version is written to [dspace]/config. +You probably won't need to edit this; the install-configs +script fills out the relevant deployment-specific parameters. You might +want to change the earliestDatestamp +field to accurately reflect the oldest datestamp in the system. (Note +that this is the value of the last_modified +column in the Item +database table.)

+
+
oai-web.xml
+
+

This standard Java Servlet +'deployment descriptor' is stored in the source as [dspace-source]/etc/oai-web.xml, +and is written to /dspace/oai/WEB-INF/web.xml.

+
+
+

Sets

+

OAI-PMH allows repositories to +expose an hierarchy of sets in which records may be placed. A record +can be in zero or more sets.

+

DSpace exposes collections as +sets. The organization of communities is likely to change over time, +and is therefore a less stable basis for selective harvesting.

+

Each collection has a +corresponding OAI set, discoverable by harvesters via the ListSets +verb. The setSpec is the Handle of the collection, with the ':' and '/' +converted to underscores so that the Handle is a legal setSpec, for +example:

+
hdl_1721.1_1234
+

Naturally enough, the +collection name is also the name of the corresponding set.

+

Unique Identifier

+

Every item in OAI-PMH data +repository must have an unique identifier, which must conform to the +URI syntax. As of DSpace 1.2, Handles are not used; this is because in +OAI-PMH, the OAI identifier identifies the metadata +record associated with the resource. +The resource +is the DSpace item, whose resource +identifier is the Handle. In +practical terms, using the Handle for the OAI identifier may cause +problems in the future if DSpace instances share items with the same +Handles; the OAI metadata record identifiers should be different as the +different DSpace instances would need to be harvested separately and +may have different metadata for the item.

+

The OAI identifiers that DSpace +uses are of the form:

+

oai:host +name:handle

+

For example:

+

oai:dspace.myu.edu:123456789/345

+

If you wish to use a different +scheme, this can easily be changed by editing the value of OAI_ID_PREFIX +at the top of the org.dspace.app.oai.DSpaceOAICatalog +class. (You do not need to change the code if the above scheme works +for you; the code picks up the host name and Handles automatically from +the DSpace configuration.)

+

Access control

+

OAI provides no +authentication/authorisation details, although these could be +implemented using standard HTTP methods. It is assumed that all access +will be anonymous for the time being.

+

A question is, "is all metadata +public?" Presently the answer to this is yes; all metadata is exposed +via OAI-PMH, even if the item has restricted access policies. The +reasoning behind this is that people who do actually have permission to +read a restricted item should still be able to use OAI-based services +to discover the content.

+

If in the future, this 'expose +all metadata' approach proves unsatisfactory for any reason, it should +be possible to expose only publicly readable metadata. The +authorisation system has separate permissions for READing and item and +READing the content (bitstreams) within it. This means the system can +differentiate between an item with public metadata and hidden content, +and an item with hidden metadata as well as hidden content. In this +case the OAI data repository should only expose items those with +anonymous READ access, so it can hide the existence of records to the +outside world completely. In this scenario, one should be wary of +protected items that are made public after a time. When this happens, +the items are "new" from the OAI-PMH perspective.

+

Modification Date (OAI Date +Stamp)

+

OAI-PMH harvesters need to know +when a record has been created, changed or deleted. DSpace keeps track +of a 'last modified' date for each item in the system, and this date is +used for the OAI-PMH date stamp. This means that any changes to the +metadata (e.g. admins correcting a field, or a withdrawal) will be +exposed to harvesters.

+

'About' Information

+

As part of each record given +out to a harvester, there is an optional, repeatable "about" section +which can be filled out in any (XML-schema conformant) way. Common uses +are for provenance and rights information, and there are schemas in use +by OAI communities for this. Presently DSpace does not provide any of +this information.

+

Deletions

+

DSpace keeps track of deletions +(withdrawals). These are exposed via OAI, which has a specific +mechansim for dealing with this. Since DSpace keeps a permanent record +of withdrawn items, in the OAI-PMH sense DSpace supports deletions +'persistently'. This is as opposed to 'transient' deletion support, +which would mean that deleted records are forgotten after a time.

+

Once an item has been +withdrawn, OAI-PMH harvests of the date range in which the withdrawal +occurred will find the 'deleted' record header. Harvests of a date +range prior to the withdrawal will not +find the record, despite the fact that the record did exist at that +time.

+

As an example of this, consider +an item that was created on 2002-05-02 and withdrawn on 2002-10-06. A +request to harvest the month 2002-10 will yield the 'record deleted' +header. However, a harvest of the month 2002-05 will not yield the +original record.

+

Note that presently, the +deletion of 'expunged' items is not exposed through OAI.

+

Flow Control (Resumption +Tokens)

+

An OAI data provider can +prevent any performance impact caused by harvesting by forcing a +harvester to receive data in time-separated chunks. If the data +provider receives a request for a lot of data, it can send part of the +data with a resumption token. The harvester can then return later with +the resumption token and continue.

+

DSpace supports resumption +tokens for 'ListRecords' OAI-PMH requests. ListIdentifiers and ListSets +requests do not produce a particularly high load on the system, so +resumption tokens are not used for those requests.

+

Each OAI-PMH ListRecords +request will return at most 100 records. This limit is set at the top +of org.dspace.app.oai.DSpaceOAICatalog.java +(MAX_RECORDS). +A potential issue here is that if a harvest yields an exact multiple of +MAX_RECORDS, +the last operation will result in a harvest with no records in it. It +is unclear from the OAI-PMH specification if this is acceptable.

+

When a resumption token is +issued, the optional completeListSize +and cursor +attributes are not included. OAICat sets the expirationDate +of the resumption token to one hour after it was issued, though in fact +since DSpace resumption tokens contain all the information required to +continue a request they do not actually expire.

+

Resumption tokens contain all +the state information required to continue a request. The format is:

+
from/until/setSpec/offset
+

from +and until +are the ISO 8601 dates passed in as part of the original request, and setSpec +is also taken from the original request. offset +is the number of records that have already been sent to the harvester. +For example:

+
2003-01-01//hdl_1721_1_1234/300
+

This means the harvest is +'from' 2003-01-01, +has no 'until' date, is for collection hdl:1721.1/1234, and 300 records +have already been sent to the harvester. (Actually, if the original +OAI-PMH request doesn't specify a 'from' or 'until, OAICat fills them +out automatically to '0000-00-00T00:00:00Z' and '9999-12-31T23:59:59Z' +respectively. This means DSpace resumption tokens will always have from +and until dates in them.)

+

Item Importer and Exporter

+

DSpace has a set of command +line tools for importing and exporting items in batches, using the +DSpace simple archive format. The tools are not terribly robust, but +are useful and are easily modified. They also give a good demonstration +of how to implement your own item importer if desired.

+

Warning: templates may be +applied

+

Due to a bug as of 1.2 beta 2, +if you have an Item template in your Collection, then those default +values may be added to Items that you import. Be sure to remove the +template if this is unwanted behavior.

+

DSpace simple archive format

+

The basic concept behind the +DSpace's simple archive format is to create an archive, which is +directory full of items, with a subdirectory per item. Each item +directory contains a file for the item's descriptive metadata, and the +files that make up the item.

+
archive_directory/
    item_000/
        dublin_core.xml -- qualified Dublin Core metadata
        contents        -- text file containing one line per filename
        file_1.doc      -- files to be added as bitstreams to the item
        file_2.pdf
    item_001/
        dublin_core.xml
        contents
        file_1.png
        ...
+

The dublin_core.xml +file has the following format, where each Dublin Core element has it's +own entry within a <dcvalue> +tagset. There are currently three tag elements available in the <dcvalue> +tagset:

+ +
<dublin_core>
    <dcvalue element="title" qualifier="none">A Tale of Two Cities</dcvalue>
    <dcvalue element="date" qualifier="issued">1990</dcvalue></dublin_core>
    <dcvalue element="title" qualifier="alternate" language="fr" ">J'aime les Printemps</dcvalue>
</dublin_core>
+

(Note the optional language tag +which notifies the system that the optional title is in French.)

+

Importing Items

+

Note: +Before running the item importer over items previously exported from a +DSpace instance, please first refer +to Transferring +Items Between DSpace Instances.

+

The item importer is in org.dspace.app.itemimport.ItemImport, +and is run with the +dsrun +utility in the dspace/bin +directory. Running it with -h gets the current command-line arguments. +Another very important flag is the --test flag, which you can use with +any command to simulate all of the actions it will perform without +actually making any changes to your DSpace instance - very useful for +validating your item directories before doing an import. In the +importer's arguments you can use either the user's database ID or email +address and the eperson ID, and the collection's database ID or handle +as arguments. Currently with the importer you can add, remove, and +replace items in a collection. If you specify more than one collection +argument then the items will be imported to multiple collections, and +the first collection specified becomes the "owning" collection. If +there is an error and the import is aborted, there is a --resume flag +that you can try to resume the import where you left off after you fix +the error.

+

To add items to a collection +with an EPerson as the submitter, type:

+
dsrun org.dspace.app.itemimport.ItemImport --add --eperson=joe@user.com  --collection=collectionID --source=items_dir --mapfile=mapfile
+

(or by using the short form)

+
dsrun org.dspace.app.itemimport.ItemImport -a -e joe@user.com  -c collectionID -s items_dir -m mapfile
+

which would then cycle through +the archive directory's items, import them, and then generate a map +file which stores the mapping of item directories to item handles. Save +this map file! Using the map file you can then 'unimport' with the +command:

+
dsrun org.dspace.app.itemimport.ItemImport --delete --mapfile=mapfile
+

The imported items listed in +the map file would then be deleted. If you wish to replace previously +imported items, you can give the command:

+
dsrun org.dspace.app.itemimport.ItemImport --replace --eperson=joe@user.com --collection=collectID --source=items_dir --mapfile=mapfile
+

Replacing items uses the map +file to replace the old items and still retain their handles.

+

The importer usually bypasses +any workflow assigned to a collection, but adding the --workflow option +will route the imported items through the workflow system.

+

The importer also has a --test +flag that will simulate the entire import process without actually +doing the import. This is extremely useful for verifying your import +files before doing the import step.

+

Exporting +Items

+

The item exporter can export a +single item or a collection of items, and creates a DSpace simple +archive for each item to be exported. To export a collection's items +you type:

+
dsrun org.dspace.app.itemexport.ItemExport --type=COLLECTION --id=collID --dest=dest_dir --number=seq_num
+

The keyword COLLECTION +means that you intend to export an entire collection. The ID can either +be the database ID or the handle. The exporter will begin numbering the +simple archives with the sequence number that you supply. To export a +single item use the keyword ITEM +and give the item ID as an argument:

+
dsrun org.dspace.app.itemexport.ItemExport --type=ITEM --id=itemID --dest=dest_dir --number=seq_num
+

Each exported item will have an +additional file in its directory, named 'handle'. This will contain the +handle that was assigned to the item, and this file will be read by the +importer so that items exported and then imported to another machine +will retain the item's original handle.

+

Transferring +Items Between DSpace +Instances

+

Where items are to be moved +between DSpace instances (for example from a test DSpace into a +production DSpace) the item exporter and item importer can be used in +conjunction with a script to assist in this process.

+

After running the item exporter +each dublin_core.xml +file will contain metadata that was automatically added by DSpace. +These fields are as follows:

+ +

In order to avoid duplication +of this metadata, run

+

dspace_migrate +<exported item directory>

+

+

prior to running the item +importer. This will remove the above metadata items from the dublin_core.xml +file and remove all handle +files. It will then be safe to run the item exporter. Use

+

dspace_migrate +--help

+

for instructions on use of the +script.

+

Registration

+

Registration is an alternate +means of incorporating items, their metadata, and their bitstreams into +DSpace by taking advantage of the bitstreams already being in +storage accessible to DSpace. An example might be that there is a +repository for existing digital assets. Rather than using the normal +interactive +ingest process or the batch +import +to furnish DSpace the metadata +and to upload bitstreams, registration provides DSpace the metadata and +the location +of the +bitstreams. DSpace uses a variation of the import tool to accomplish +registration.

+

Accessible Storage

+

To register an item its +bitstreams must reside on storage accessible to DSpace and therefore +referenced by an asset store number +in dspace.cfg. +The configuration +file +dspace.cfg +establishes one or more asset stores through the use of an integer +asset store number. This number relates to a directory in the DSpace +host's file system or a set of SRB account parameters. This asset store +number is described in The dspace.cfg +Configuration Properties File +section and in the dspace.cfg +file itself. The asset store number(s) used for registered items should +generally not be the value of the assetstore.incoming +property since it is +unlikely that that you will want to mix the bitstreams of normally +ingested and imported items and registered items.

+

Registering Items Using the +Item Importer

+

DSpace uses the same import +tool that is used for batch import except +that several variations are employed to support registration. The +discussion that follows assumes familiarity with the import tool.

+

The archive format for +registration does not include the actual content +files (bitstreams) being registered. The format is however a directory +full of items to be registered, with a subdirectory per item. Each item +directory contains a file for the item's descriptive metadata +(dublin_core.xml) +and a file listing the item's content files +(contents), +but not the actual content files themselves.

+

The dublin_core.xml +file for +item registration is exactly the same as +for regular item import.

+

The contents +file, like that +for regular item import, lists the item's +content files, one content file per line, but each line has the one of +the following formats:

+
-r -s n -f filepath
+
-r -s n -f filepath\tbundle:bundlename
+where +

-r +indicates this is a file to +be +registered
+-s n +indicates the asset store +number (n)
+-f filepath +indicates the path +and name of the content file to be registered (filepath)
+\t +is a tab character
+bundle:bundlename +is an +optional bundle +name

+

+The bundle, that is everything after the filepath, is optional and is +normally not used.

+

The command line for +registration is just like the one for regular +import:

+
dsrun org.dspace.app.itemimport.ItemImport --add --eperson=joe@user.com  --collection=collectionID --source=items_dir --mapfile=mapfile
+

(or by using the short form)

+
dsrun org.dspace.app.itemimport.ItemImport -a -e joe@user.com  -c collectionID -s items_dir -m mapfile
+

The --workflow +and --test flags +will function as described in Importing +Items.

+

The --delete flag will function +as described in Importing +Items +but the registered content files will not be removed from storage. See Deleting Registered +Items.

+

The --replace flag will +function as described in Importing +Items +but care should be taken to consider different cases and implications. +With old items and new items being registered or ingested normally, +there are four combinations or cases to consider. Foremost, an old +registered item deleted from DSpace using --replace will +not be removed +from the storage. See Deleting +Registered Items. where is +resides. A new item added to DSpace using --replace will +be ingested +normally or will be registered depending on whether or not it is marked +in the contents files with the -r.

+

Internal Identification +and Retrieval +of Registered Items

+

Once an item has been +registered, superficially it is indistinguishable +from items ingested interactively or by batch import. But internally +there are some differences:

+

First, the randomly generated +internal ID is not used because DSpace +does not control the file path and name of the bitstream. Instead, the +file path and name are that specified in the contents +file.

+

Second, the store_number +column of the bitstream database +row contains +the asset store number specified in the contents +file.

+

Third, the internal_id +column of the bitstream database +row contains a +leading flag (-R) +followed by the registered file path and name. +For example, +-Rfilepath +where filepath +is the file path +and name relative to the +asset store corresponding to the asset store number. The asset store +could be traditional storage in the DSpace server's file system or an +SRB account.

+

Fourth,  an MD5 +checksum is calculated by reading the registered file if it is in local +storage. If the registerd file is in remote storage (say, SRB) a +checksup is calulated on just the file name! This is an efficiency +choice since registering a large number of large files that +are in SRB would consume substantial network resources and time. A +future option could be to have an SRB proxy process calculate MD5s and +store them in SRB's metadata catalog (MCAT) for rapid retrieval. SRB +offers such an option but it's not yet in production release.

+

Registered items and their +bitstreams can be retrieved transparently +just like normally ingested items.

+

Exporting Registered Items +

+

Registered items may be +exported as described in Exporting Items. +If so, the export directory will contain actual copies of the files +being exported but the lines in the contents file will flag the files +as registered. This means that if DSpace items are "round tripped" (see +Transferring Items Between DSpace Instances) using the exporter and +importer, the registered files in the export directory will again +registered in DSpace instead of being uploaded and ingested normally.

+

METS Export of Registered +Items +

+

The METS Export Tool +can also be used but note the cautions described in that section and +note that MD5 values for items in remote storage are actually MD5 +values on just the file name.

+

Deleting +Registered Items

+

If a registered item is deleted +from DSpace, either interactively or by +using the --delete or --replace flags +described in Importing Items, +the item will disappear from DSpace but it's registered content files +will remain in place just as they were prior to registration. +Bitstreams not registered but added by DSpace as part of registration, +such as license.txt files, will be deleted.

+

METS Tools

+

The experimental (incomplete) +METS export tool writes DSpace items to a filesystem with the metadata +held in a more standard format based on METS.

+

The Export Tool

+

The METS export tool is invoked +via the command line like this:

+
[dspace]/bin/dsrun org.dspace.app.mets.METSExport --help
+

The tool can export an +individual +item, the items within a given collection, or everything in +the DSpace instance. To export an individual item, use:

+
[dspace]/bin/dsrun org.dspace.app.mets.METSExport --item [handle]
+

To export the items in +collection hdl:123.456/789, +use:

+
[dspace]/bin/dsrun org.dspace.app.mets.METSExport --collection hdl:123.456/789
+

To export all the items DSpace, +use:

+
[dspace]/bin/dsrun org.dspace.app.mets.METSExport --all
+

With any of the above forms, +you can specify the base directory into which the items will be +exported, using --destination +[directory]. +If this parameter is omitted, the current directory is used.

+

The AIP Format

+

Each exported item is written +to a separate directory, created under the base directory specified in +the command-line arguments, or in the current directory if --destination +is omitted. The name of each directory is the Handle, URL-encoded so +that the directory name is 'legal'.

+

Within each item directory is a +mets.xml +file which contains the METS-encoded metadata for the item. Bitstreams +in the item are also stored in the directory. Their filenames are their +MD5 checksums, firstly for easy integrity checking, and also to avoid +any problems with 'special characters' in the filenames that were legal +on the original filing system they came from but are illegal in the +server filing system. The mets.xml +file includes XLink pointers to these bitstream files.

+

An example AIP might look like +this:

+ +

The contents of the METS in the +mets.xml +file are as follows:

+ +

Limitations

+ +

MediaFilters: Transforming DSpace +Content

+

DSpace can apply filters to +content/bitstreams, creating new content. Filters are included that +extract text for full-text +searching, and create thumbnails +for items that contain images. The media filters are controlled by the MediaFilterManager +which traverses the asset store, invoking the MediaFilter +subclasses on bitstreams. The file config/mediafilter.cfg contains a +list of bitstream format types and the filters that operate on +bitstreams of that type. The media filter system is intended to be run +from the command line (or regularly as a cron task):

+
dspace/bin/filter-media
+

Traverse the asset store, +applying media filters to bitstreams, skipping bitstreams that have +already been filtered.

+
dspace/bin/filter-media -f
+

Apply filters to ALL +bitstreams, even if they've already been filtered.

+
dspace/bin/filter-media -v
+

Verbose mode - print all +extracted text and other filter details to STDOUT.

+
dspace/bin/filter-media -n
+

Suppress index creation - by +default, a new search index is created for full-text searching. This +option suppresses index creation if you intend to run index-all +elsewhere.

+

Adding your own filters is done +by creating a sub-class of the MediaFilter +class. See the comments in the source file MediaFilter.java for more +information. In theory filters could be implemented in any language (C, +Perl, etc.) They only need to be invoked by the Java code in the MediaFilter +class that you create.

+

Sub-Community Management

+

DSpace provides an +administrative tool - 'CommunityFiliator' - for managing community +sub-structure. Normally this structure seldom changes, but prior to the +1.2 release sub-communities were not supported, so this tool could be +used to place existing pre-1.2 communities into a hierarchy. It has two +operations, either establishing a community to sub-community +relationship, or dis-establishing an existing relationship.

+

The familiar parent/child +metaphor can be used to explain how it works. Every community in DSpace +can be either a 'parent' community - meaning it has at least one +sub-community, or a 'child' community - meaning it is a sub-community +of another community, or both or neither. In these terms, an 'orphan' +is a community that lacks a parent (although it can be a parent); +'orphans' are referred to as 'top-level' communities in the DSpace +user-interface, since there is no parent community 'above' them. The +first operation - establishing a parent/child relationship - can take +place between any community and an orphan. The second operation - +removing a parent/child relationship - will make the child an orphan.

+

Using the dsrun utility in the +dspace/bin directory, the establish operation looks like this:

+
dsrun org.dspace.administer.CommunityFiliator --set --parent=parentID --child=childID
+

(or using the short form)

+
dsrun org.dspace.administer.CommunityFiliator -s -p parentID -c childID
+

where '-s' or '--set' means +establish a relationship whereby the community identified by the '-p' +parameter becomes the parent of the community identified by the '-c' +parameter. Both the 'parentID' and 'childID' values may be handles or +database IDs.

+

The reverse operation looks +like this:

+
dsrun org.dspace.administer.CommunityFiliator --remove --parent=parentID --child=childID
+

(or using the short form)

+
dsrun org.dspace.administer.CommunityFiliator -r -p parentID -c childID
+

where '-r' or '--remove' means +dis-establish the current relationship in which the community +identified by 'parentID' is the parent of the community identified by +'childID'. The outcome will be that the 'childID' community will become +an orphan, i.e. a top-level community.

+

If the required constraints of +operation are violated, an error message will appear explaining the +problem, and no change will be made. An example in a removal operation, +where the stated child community does not have the stated parent +community as its parent: "Error, child community not a child of parent +community".

+

It is possible to effect +arbitrary changes to the community hierarchy by chaining the basic +operations together. For example, to move a child community from one +parent to another, simply perform a 'remove' from its current parent +(which will leave it an orphan), followed by a 'set' to its new parent.

+

It is important to understand +that when any operation is performed, all the sub-structure of the +child community follows it. Thus, if a child has itself children +(sub-communities), or collections, they will all move with it to its +new 'location' in the community tree.

+
+
Copyright © +2002-2004 MIT and Hewlett Packard
+ + diff --git a/dspace/docs/architecture.html b/dspace/docs/architecture.html new file mode 100644 index 0000000000..a8caf90d06 --- /dev/null +++ b/dspace/docs/architecture.html @@ -0,0 +1,137 @@ + + + + DSpace System +Documentation: Architecture + + + + +

DSpace +System Documentation: Architecture

+

Back to contents

+

Overview

+

The DSpace system is organized +into three layers, each of which +consists of a number of components.

+

Application Layer, Business Logic Layer, Storage Layer

+

DSpace +System Architecture

+

The storage layer is +responsible for physical storage of metadata +and content. The business logic layer deals with managing the content +of the archive, users of the archive (e-people), authorization, and +workflow. The application layer contains components that communicate +with the world outside of the individual DSpace installation, for +example the Web user interface and the Open Archives +Initiative +protocol for metadata harvesting service.

+

Each layer only invokes the +layer below it; the application layer +may not used the storage layer directly, for example. Each component in +the storage and business logic layers has a defined public API. The +union of the APIs of those components are referred to as the Storage +API (in the case of the storage layer) and the DSpace Public API (in +the case of the business logic layer). These APIs are in-process Java +classes, objects and methods.

+

It is important to note that +each layer is trusted. +Although the logic for authorising +actions is in the business +logic layer, the system relies on individual applications in the +application layer to correctly and securely authenticate +e-people. If a 'hostile' or insecure application were allowed to invoke +the Public API directly, it could very easily perform actions as any +e-person in the system.

+

The reason for this design +choice is that authentication methods +will vary widely between different applications, so it makes sense to +leave the logic and responsibility for that in these applications.

+

The source code is organized to +cohere very strictly to this +three-layer architecture. Also, only methods in a component's public +API are given the public +access level. This means that +the Java compiler helps ensure that the source code conforms to the +architecture.

+ + + + + + + + + + + + + + + + + + + +
Source Code Packages
Packages withinCorrespond to components +in
org.dspace.appApplication layer
org.dspaceBusiness logic layer +(except storage +and app)
org.dspace.storageStorage layer
+

The storage and business logic +layer APIs are extensively documented +with Javadoc-style comments. Generate the HTML version of these by +entering the source directory and running:

+
ant public_api
+

The package-level documentation +of each package usually contains an +overview of the package and some example usage. This information is not +repeated in this architecture document; this and the Javadoc APIs are +intended to be used in parallel.

+

Each layer is described in a +separate section:

+ +
+
Copyright © +2002-2004 MIT and Hewlett Packard
+ + diff --git a/dspace/docs/business.html b/dspace/docs/business.html new file mode 100644 index 0000000000..4720bbbc33 --- /dev/null +++ b/dspace/docs/business.html @@ -0,0 +1,773 @@ + + + + DSpace System Documentation: Business Logic Layer + + + + +

DSpace System Documentation: Business Logic Layer

+ +

Back to contents
Back to architecture overview

+ +

Core Classes

+ +

The org.dspace.core package provides some basic classes that are used throughout the DSpace code.

+ +

The Configuration Manager (ConfigurationManager)

+ +

The configuration manager is responsible for reading the main dspace.cfg properties file, managing the 'template' configuration files for other applications such as Apache, and for obtaining the text for e-mail messages.

+ +

The system is configured by editing the relevant files in /dspace/config, as described in the configuration section.

+ +

When editing configuration files for applications that DSpace uses, such as Apache, remember to edit the file in /dspace/config/templates and then run /dspace/bin/install-configs rather than editing the 'live' version directly!

+ +

The ConfigurationManager class can also be invoked as a command line tool, with two possible uses:

+ + + + +

Constants

+ +

This class contains constants that are used to represent types of object and actions in the database. For example, authorization policies can relate to objects of different types, so the resourcepolicy table has columns resource_id, which is the internal ID of the object, and resource_type_id, which indicates whether the object is an item, collection, bitstream etc. The value of resource_type_id is taken from the Constants class, for example Constants.ITEM.

+ + +

Context

+ +

The Context class is central to the DSpace operation. Any code that wishes to use the any API in the business logic layer must first create itself a Context object. This is akin to opening a connection to a database (which is in fact one of the things that happens.)

+ +

A context object is involved in most method calls and object constructors, so that the method or object has access to information about the current operation. When the context object is constructed, the following information is automatically initialized:

+ + + +

The following information is also held in a context object, though it is the responsiblity of the application creating the context object to fill it out correctly:

+ + + +

Typical use of the context object will involve constructing one, and setting the current user if one is authenticated. Several operations may be performed using the context object. If all goes well, complete is called to commit the changes and free up any resources used by the context. If anything has gone wrong, abort is called to roll back any changes and free up the resources.

+ +

You should always abort a context if any error happens during its lifespan; otherwise the data in the system may be left in an inconsistent state. You can also commit a context, which means that any changes are written to the database, and the context is kept active for further use.

+ + +

Email

+ +

Sending e-mails is pretty easy. Just use the configuration manager's getEmail method, set the arguments and recipients, and send.

+ +

The e-mail texts are stored in /dspace/config/emails. They are processed by the standard java.text.MessageFormat. At the top of each e-mail are listed the appropriate arguments that should be filled out by the sender. Example usage is shown in the org.dspace.core.Email Javadoc API documentation.

+ + +

LogManager

+ +

The log manager consists of a method that creates a standard log header, and returns it as a string suitable for logging. Note that this class does not actually write anything to the logs; the log header returned should be logged directly by the sender using an appropriate Log4J call, so that information about where the logging is taking place is also stored.

+ +

The level of logging can be configured on a per-package or per-class basis by editing /dspace/config/templates/log4j.properties and then executing /dspace/bin/install-configs. You will need to stop and restart Tomcat for the changes to take effect.

+ +

A typical log entry looks like this:

+ +

2002-11-11 08:11:32,903 INFO org.dspace.app.webui.servlet.DSpaceServlet @ anonymous:session_id=BD84E7C194C2CF4BD0EC3A6CAD0142BB:view_item:handle=1721.1/1686

+ +

This is breaks down like this:

+ + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + +
Date and time, milliseconds2002-11-11 08:11:32,903
Level (FATAL, WARN, INFO or DEBUG)INFO
Java classorg.dspace.app.webui.servlet.DSpaceServlet
@
User email or anonymousanonymous
:
Extra log info from contextsession_id=BD84E7C194C2CF4BD0EC3A6CAD0142BB
:
Actionview_item
:
Extra infohandle=1721.1/1686
+ +

The above format allows the logs to be easily parsed and analysed. The /dspace/bin/log-reporter script is a simple tool for analysing logs. Try:

+ + 
/dspace/bin/log-reporter --help
+ +

It's a good idea to 'nice' this log reporter to avoid an impact on server performance.

+ + +

Utils

+ +

Utils comtains miscellaneous utility method that are required in a variety of places throughout the code, and thus have no particular 'home' in a subsystem.

+ + +

Content Management API

+ +

The content management API package org.dspace.content contains Java classes for reading and manipulating content stored in the DSpace system. This is the API that components in the application layer will probably use most.

+ +

Classes corresponding to the main elements in the DSpace data model (Community, Collection, Item, Bundle and Bitstream) are sub-classes of the abstract class DSpaceObject. The Item object handles the Dublin Core metadata record.

+ +

Each class generally has one or more static find methods, which are used to instantiate content objects. Constructors do not have public access and are just used internally. The reasons for this are:

+ + + +

Collection, Bundle and Bitstream do not have create methods; rather, one has to create an object using the relevant method on the container. For example, to create a collection, one must invoke createCollection on the community that the collection is to appear in:

+ + 
Context context = new Context();
+Community existingCommunity = Community.find(context, 123);
+Collection myNewCollection = existingCommunity.createCollection();
+ +

The primary reason for this is for determining authorization. In order to know whether an e-person may create an object, the system must know which container the object is to be added to. It makes no sense to create a collection outside of a community, and the authorization system does not have a policy for that.

+ +

Items are first created in the form of an implementation of InProgressSubmission. An InProgressSubmission represents an item under construction; once it is complete, it is installed into the main archive and added to the relevant collection by the InstallItem class. The org.dspace.content package provides an implementation of InProgressSubmission called WorkspaceItem; this is a simple implementation that contains some fields used by the Web submission UI. The org.dspace.workflow also contains an implementation called WorkflowItem which represents a submission undergoing a workflow process.

+ +

In the previous chapter there is an overview of the item ingest process which should clarify the previous paragraph. Also see the section on the workflow system.

+ +

Community and BitstreamFormat do have static create methods; one must be a site administrator to have authorization to invoke these.

+ + +

Other Classes

+ +

Classes whose name begins DC are for manipulating Dublin Core metadata, as explained below.

+ +

The FormatIdentifier class attempts to guess the bitstream format of a particular bitstream. Presently, it does this simply by looking at any file extension in the bitstream name and matching it up with the file extensions associated with bitstream formats. Hopefully this can be greatly improved in the future!

+ +

The ItemIterator class allows items to be retrieved from storage one at a time, and is returned by methods that may return a large number of items, more than would be desirable to have in memory at once.

+ +

The ItemComparator class is an implementation of the standard java.util.Comparator that can be used to compare and order items based on a particular Dublin Core metadata field.

+ + + +

Modifications

+ +

When creating, modifying or for whatever reason removing data with the content management API, it is important to know when changes happen in-memory, and when they occur in the physical DSpace storage.

+ +

Primarily, one should note that no change made using a particular org.dspace.core.Context object will actually be made in the underlying storage unless complete or commit is invoked on that Context. If anything should go wrong during an operation, the context should always be aborted by invoking abort, to ensure that no inconsistent state is written to the storage.

+ +

Additionally, some changes made to objects only happen in-memory. In these cases, invoking the update method lines up the in-memory changes to occur in storage when the Context is committed or completed. In general, methods that change any [meta]data field only make the change in-memory; methods that involve relationships with other objects in the system line up the changes to be committed with the context. See individual methods in the API Javadoc.

+ +

Some examples to illustrate this are shown below:

+ + + + + + + + + + + + + + + + + + + 
Context context = new Context();
+Bitstream b = Bitstream.find(context, 1234);
+b.setName("newfile.txt");
+b.update();
+context.complete();
+ 		Will change storage
Context context = new Context();
+Bitstream b = Bitstream.find(context, 1234);
+b.setName("newfile.txt");
+b.update();
+context.abort();
+ 		Will not change storage (context aborted)
Context context = new Context();
+Bitstream b = Bitstream.find(context, 1234);
+b.setName("newfile.txt");
+context.complete();
+ 		The new name will not be stored since update was not invoked
Context context = new Context();
+Bitstream bs = Bitstream.find(context, 1234);
+Bundle bnd = Bundle.find(context, 5678);
+bnd.add(bs);
+context.complete();
+ 		The bitstream will be included in the bundle, since update doesn't need to be called
+ + +

What's In Memory?

+ +

Instantiating some content objects also causes other content objects to be loaded into memory.

+ +

Instantiating a Bitstream object causes the appropriate BitstreamFormat object to be instantiated. Of course the Bitstream object does not load the underlying bits from the bitstream store into memory!

+ +

Instantiating a Bundle object causes the appropriate Bitstream objects (and hence BitstreamFormats) to be instantiated.

+ +

Instantiating an Item object causes the appropriate Bundle objects (etc.) and hence BitstreamFormats to be instantiated. All the Dublin Core metadata associated with that item are also loaded into memory.

+ +

The reasoning behind this is that for the vast majority of cases, anyone instantiating an item object is going to need information about the bundles and bitstreams within it, and this methodology allows that to be done in the most efficient way and is simple for the caller. For example, in the Web UI, the servlet (controller) needs to pass information about an item to the viewer (JSP), which needs to have all the information in-memory to display the item without further accesses to the database which may cause errors mid-display.

+ +

You do not need to worry about multiple in-memory instantiations of the same object, or any inconsistenties that may result; the Context object keeps a cache of the instantiated objects. The find methods of classes in org.dspace.content will use a cached object if one exists.

+ +

It may be that in enough cases this automatic instantiation of contained objects reduces performance in situations where it is important; if this proves to be true the API may be changed in the future to include a loadContents method or somesuch, or perhaps a Boolean parameter indicating what to do will be added to the find methods.

+ +

When a Context object is completed, aborted or garbage-collected, any objects instantiated using that context are invalidated and should not be used (in much the same way an AWT button is invalid if the window containing it is destroyed).

+ + +

Dublin Core Metadata

+ +

The DCValue class is a simple container that represents a single Dublin Core element, optional qualifier, value and language. The other classes starting with DC are utility classes for handling types of data in Dublin Core, such as people's names and dates. As supplied, the DSpace registry of elements and qualifiers corresponds to the Library Application Profile for Dublin Core. It should be noted that these utility classes assume that the values will be in a certain syntax, which will be true for all data generated within the DSpace system, but since Dublin Core does not always define strict syntax, this may not be true for Dublin Core originating outside DSpace.

+ +

Below is the specific syntax that DSpace expects various fields to adhere to:

+ + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + +
ElementQualifierSyntaxHelper Class
dateAny or unqualified +

ISO 8601 in the UTC time zone, with either year, month, day, or second precision. Examples:

+ 
2000
+2002-10
+2002-08-14
+1999-01-01T14:35:23Z
+ 		DCDate
contributorAny or unqualified +

In general last name, then a comma, then first names, then any additional information like "Jr.". If the contributor is an organization, then simply the name. Examples:

+ 
Doe, John
+Smith, John Jr.
+van Dyke, Dick
+Massachusetts Institute of Technology
+ 		DCPersonName
languageiso +

A two letter code taken ISO 639, followed optionally by a two letter country code taken from ISO 3166. Examples:

+ 
en
+fr
+en_US
+ 		DCLanguage
relationispartofseries +

The series name, following by a semicolon followed by the number in that series. Alternatively, just free text.

+ 
MIT-TR; 1234
+My Report Series; ABC-1234
+NS1234
+ 		DCSeriesNumber
+ + +

Workflow System

+ +

The primary classes are:

+ + + + + + + + + + + + + + + + + + + + + + + + + + +
org.dspace.content.WorkspaceItemcontains an Item before it enters a workflow
org.dspace.workflow.WorkflowItemcontains an Item while in a workflow
org.dspace.workflow.WorkflowManagerresponds to events, manages the WorkflowItem states
org.dspace.content.Collectioncontains List of defined workflow steps
org.dspace.eperson.Grouppeople who can perform workflow tasks are defined in EPerson Groups
org.dspace.core.Emailused to email messages to Group members and submitters
+ +

The workflow system models the states of an Item in a state machine with 5 states (SUBMIT, STEP_1, STEP_2, STEP_3, ARCHIVE.) These are the three optional steps where the item can be viewed and corrected by different groups of people. Actually, it's more like 8 states, with STEP_1_POOL, STEP_2_POOL, and STEP_3_POOL. These pooled states are when items are waiting to enter the primary states.

+ +

The WorkflowManager is invoked by events. While an Item is being submitted, it is held by a WorkspaceItem. Calling the start() method in the WorkflowManager converts a WorkspaceItem to a WorkflowItem, and begins processing the WorkflowItem's state. Since all three steps of the workflow are optional, if no steps are defined, then the Item is simply archived.

+ +

Workflows are set per Collection, and steps are defined by creating corresponding entries in the List named workflowGroup. If you wish the workflow to have a step 1, use the administration tools for Collections to create a workflow Group with members who you want to be able to view and approve the Item, and the workflowGroup[0] becomes set with the ID of that Group.

+ +

If a step is defined in a Collection's workflow, then the WorkflowItem's state is set to that step_POOL. This pooled state is the WorkflowItem waiting for an EPerson in that group to claim the step's task for that WorkflowItem. The WorkflowManager emails the members of that Group notifying them that there is a task to be performed (the text is defined in config/emails,) and when an EPerson goes to their 'My DSpace' page to claim the task, the WorkflowManager is invoked with a claim event, and the WorkflowItem's state advances from STEP_x_POOL to STEP_x (where x is the corresponding step.) The EPerson can also generate an 'unclaim' event, returning the WorkflowItem to the STEP_x_POOL.

+ +

Other events the WorkflowManager handles are advance(), which advances the WorkflowItem to the next state. If there are no further states, then the WorkflowItem is removed, and the Item is then archived. An EPerson performing one of the tasks can reject the Item, which stops the workflow, rebuilds the WorkspaceItem for it and sends a rejection note to the submitter. More drastically, an abort() event is generated by the admin tools to cancel a workflow outright.

+ + +

Administration Toolkit

+ +

The org.dspace.administer package contains some classes for administering a DSpace system that are not generally needed by most applications.

+ +

The CreateAdministrator class is a simple command-line tool, executed via /dspace/bin/create-administrator, that creates an administrator e-person with information entered from standard input. This is generally used only once when a DSpace system is initially installed, to create an initial administrator who can then use the Web administration UI to further set up the system. This script does not check for authorization, since it is typically run before there are any e-people to authorize! Since it must be run as a command-line tool on the server machine, generally this shouldn't cause a problem. A possibility is to have the script only operate when there are no e-people in the system already, though in general, someone with access to command-line scripts on your server is probably in a position to do what they want anyway!

+ +

The DCType class is similar to the org.dspace.content.BitstreamFormat class. It represents an entry in the Dublin Core type registry, that is, a particular element and qualifier, or unqualified element. It is in the administer package because it is only generally required when manipulating the registry itself. Elements and qualifiers are specified as literals in org.dspace.content.Item methods and the org.dspace.content.DCValue class. Only administrators may modify the Dublin Core type registry.

+ +

The org.dspace.administer.RegistryLoader class contains methods for initialising the Dublin Core type registry and bitstream format registry with entries in an XML file. Typically this is executed via the command line during the build process (see build.xml in the source.) To see examples of the XML formats, see the files in config/registries in the source directory. There is no XML schema, they aren't validated strictly when loaded in.

+ + +

E-person/Group Manager

+ +

DSpace keeps track of registered users with the org.dspace.eperson.EPerson class. The class has methods to create and manipulate an EPerson such as get and set methods for first and last names, email, and password. (Actually, there is no getPassword() method--an MD5 hash of the password is stored, and can only be verified with the checkPassword() method.) There are find methods to find an EPerson by email (which is assumed to be unique,) or to find all EPeople in the system.

+ +

The EPerson object should probably be reworked to allow for easy expansion; the current EPerson object tracks pretty much only what MIT was interested in tracking - first and last names, email, phone. The access methods are hardcoded and should probably be replaced with methods to access arbitrary name/value pairs for institutions that wish to customize what EPerson information is stored.

+ +

Groups are simply lists of EPerson objects. Other than membership, Group objects have only one other attribute: a name. Group names must be unique, so we have adopted naming conventions where the role of the group is its name, such as COLLECTION_100_ADD. Groups add and remove EPerson objects with addMember() and removeMember() methods. One important thing to know about groups is that they store their membership in memory until the update() method is called - so when modifying a group's membership don't forget to invoke update() or your changes will be lost! Since group membership is used heavily by the authorization system a fast isMember() method is also provided.

+ +

Another kind of Group is also implemented in DSpace--special Groups. The Context object for each session carries around a List of Group IDs that the user is also a member of--currently the MITUser Group ID is added to the list of a user's special groups if certain IP address or certificate criteria are met.

+ + +

Authorization

+ +

The primary classes are:

+ + + + + + + + + + + + + + +
org.dspace.authorize.AuthorizeManagerdoes all authorization, checking policies against Groups
org.dspace.authorize.ResourcePolicydefines all allowable actions for an object
org.dspace.eperson.Groupall policies are defined in terms of EPerson Groups
+ +

The authorization system is based on the classic 'police state' model of security; no action is allowed unless it is expressed in a policy. The policies are attached to resources (hence the name ResourcePolicy,) and detail who can perform that action. The resource can be any of the DSpace object types, listed in org.dspace.core.Constants (BITSTREAM, ITEM, COLLECTION, etc.) The 'who' is made up of EPerson groups. The actions are also in Constants.java (READ, WRITE, ADD, etc.) The only non-obvious actions are ADD and REMOVE, which are authorizations for container objects. To be able to create an Item, you must have ADD permission in a Collection, which contains Items. (Communities, Collections, Items, and Bundles are all container objects.)

+ +

Currently most of the read policy checking is done with items--communities and collections are assumed to be openly readable, but items and their bitstreams are checked. Separate policy checks for items and their bitstreams enables policies that allow publicly readable items, but parts of their content may be restricted to certain groups.

+ +

The AuthorizeManager class' authorizeAction(Context, object, action) is the primary source of all authorization in the system. It gets a list of all of the ResourcePolicies in the system that match the object and action. It then iterates through the policies, extracting the EPerson Group from each policy, and checks to see if the EPersonID from the Context is a member of any of those groups. If all of the policies are queried and no permission is found, then an AuthorizeException is thrown. An authorizeAction() method is also supplied that returns a boolean for applications that require higher performance.

+ +

ResourcePolicies are very simple, and there are quite a lot of them. Each can only list a single group, a single action, and a single object. So each object will likely have several policies, and if multiple groups share permissions for actions on an object, each group will get its own policy. (It's a good thing they're small.)

+ + +

Special Groups

+ +

All users are assumed to be part of the public group (ID=0.) DSpace admins (ID=1) are automatically part of all groups, much like super-users in the Unix OS. The Context object also carries around a List of special groups, which are also first checked for membership. These special groups are used at MIT to indicate membership in the MIT community, something that is very difficult to enumerate in the database! When a user logs in with an MIT certificate or with an MIT IP address, the login code adds this MIT user group to the user's Context.

+ + +

Miscellaneous Authorization Notes

+ +

Where do items get their read policies? From the their collection's read policy. There once was a separate item read default policy in each collection, and perhaps there will be again since it appears that administrators are notoriously bad at defining collection's read policies. There is also code in place to enable policies that are timed--have a start and end date. However, the admin tools to enable these sorts of policies have not been written.

+ + +

Handle Manager/Handle Plugin

+ +

The org.dspace.handle package contains two classes; HandleManager is used to create and look up Handles, and HandlePlugin is used to expose and resolve DSpace Handles for the outside world via the CNRI Handle Server code.

+ +

Handles are stored internally in the handle database table in the form:

+ + 
1721.123/4567
+ +

Typically when they are used outside of the system they are displayed in either URI or "URL proxy" forms:

+ + 
hdl:1721.123/4567
+http://hdl.handle.net/1721.123/4567
+ +

It is the responsibility of the caller to extract the basic form from whichever displayed form is used.

+ +

The handle table maps these Handles to resource type/resource ID pairs, where resource type is a value from org.dspace.core.Constants and resource ID is the internal identifier (database primary key) of the object. This allows Handles to be assigned to any type of object in the system, though as explained in the functional overview, only communities, collections and items are presently assigned Handles.

+ +

HandleManager contains static methods for:

+ + + +

HandlePlugin is a simple implementation of the Handle Server's net.handle.hdllib.HandleStorage interface. It only implements the basic Handle retrieval methods, which get information from the handle database table. The CNRI Handle Server is configured to use this plug-in via its config.dct file.

+ +

Note that since the Handle server runs as a separate JVM to the DSpace Web applications, it uses a separate 'Log4J' configuration, since Log4J does not support multiple JVMs using the same daily rolling logs. This alternative configuration is held as a template in /dspace/config/templates/log4j-handle-plugin.properties, written to /dspace/config/log4j-handle-plugin.properties by the install-configs script. The /dspace/bin/start-handle-server script passes in the appropriate command line parameters so that the Handle server uses this configuration.

+ + +

Search

+ +

DSpace's search code is a simple API which currently wraps the Lucene search engine. The first half of the search task is indexing, and org.dspace.search.DSIndexer is the indexing class, which contains indexContent() which if passed an Item, Community, or Collection, will add that content's fields to the index. The methods unIndexContent() and reIndexContent() remove and update content's index information. The DSIndexer class also has a main() method which will rebuild the index completely. This is invoked by the dspace/bin/index-all script. The intent was for the main() method to be invoked on a regular basis to avoid index corruption, but we have had no problem with that so far. Which fields are indexed by DSIndexer? These fields are currently hardcoded in indexItemContent() indexCollectionContent() and indexCommunityContent()/ methods.

+ +

The query class DSQuery contains the three flavors of doQuery() methods--one searches the DSpace site, and the other two restrict searches to Collections and Communities. The results from a query are returned as three lists of handles; each list represents a type of result. One list is a list of Items with matches, and the other two are Collections and Communities that match. This separation allows the UI to handle the types of results gracefully without resolving all of the handles first to see what kind of content the handle points to. The DSQuery class also has a main() method for debugging via command-line searches.

+ +

Our Lucene Implementation

+ +

Currently we have our own Analyzer and Tokenizer classes (DSAnalyzer and DSTokenizer) to customize our indexing. They invoke the stemming and stop word features within Lucene. We create an IndexReader for each query, which we now realize isn't the most efficient use of resources - we seem to run out of filehandles on really heavy loads. (A wildcard query can open many filehandles!) Since Lucene is thread-safe, a better future implementation would be to have a single Lucene IndexReader shared by all queries, and then is invalidated and re-opened when the index changes. Future API growth could include relevance scores (Lucene generates them, but we ignore them,) and abstractions for more advanced search concepts such as booleans.

+ +

Indexed Fields

+ +

The DSIndexer class shipped with DSpace indexes the Dublin Core metadata in the following way:

+ + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + +
Search FieldTaken from Dublin Core Fields
Authorscontributor.*
+ creator.*
+ description.statementofresponsibility
Titlestitle.*
Keywordssubject.* +
Abstractsdescription.abstract
+ description.tableofcontents
Seriesrelation.ispartofseries
MIME typesformat.mimetype
Sponsorsdescription.sponsorship
Identifiersidentifier.*
+ + + +

Harvesting API

+ +

The org.dspace.search package also provides a 'harvesting' API. This allows callers to extract information about items modified within a particular timeframe, and within a particular scope (all of DSpace, or a community or collection.) Currently this is used by the Open Archives Initiative metadata harvesting protocol application, and the e-mail subscription code.

+ +

The Harvest.harvest is invoked with the required scope and start and end dates. Either date can be omitted. The dates should be in the ISO8601, UTC time zone format used elsewhere in the DSpace system.

+ +

HarvestedItemInfo objects are returned. These objects are simple containers with basic information about the items falling within the given scope and date range. Depending on parameters passed to the harvest method, the containers and item fields may have been filled out with the IDs of communities and collections containing an item, and the corresponding Item object respectively. Electing not to have these fields filled out means the harvest operation executes considerable faster.

+ +

In case it is required, Harvest also offers a method for creating a single HarvestedItemInfo object, which might make things easier for the caller.

+ + +

Browse API

+ +

The browse API maintains indices of dates, authors and titles, and allows callers to extract parts of these:

+ +
+
Title
+ +

Values of the Dublin Core lement title (unqualified) are indexed. These are sorted in a case-insensitive fashion, with any leading article removed. For example:

+ + 
The DSpace System
+ +

Appears under 'D' rather than 'T'.

+ +
Author
+ +

Values of the contributor (any qualifier or unqualified) element are indexed. Since contributor values typically are in the form 'last name, first name', a simple case-insensitive alphanumeric sort is used which orders authors in last name order.

+ +

Note that this is an index of authors, and not items by author. If four items have the same author, that author will appear in the index only once. Hence, the index of authors may be greater or smaller than the index of titles; items often have more than one author, though the same author may have authored several items.

+ +

The author indexing in the browse API does have limitations:

+ + + +

These issues are typically resolved in libraries with authority control records, in which are kept a 'preferred' form of the author's name, with extra information (such as date of birth/death) in order to distinguish between authors of the same name. Maintaining such records is a huge task with many issues, particularly when metadata is received from faculty directly rather than trained library cataloguers. For these reasons, DSpace does not yet feature 'authority control' functionality.

+ +
Date of Issue
+ +

Items are indexed by date of issue. This may be different from the date that an item appeared in DSpace; many items may have been originally published elsewhere beforehand. The Dublin Core field used is date.issued. The ordering of this index may be reversed so 'earliest first' and 'most recent first' orderings are possible.

+ +

Note that the index is of items by date, as opposed to an index of dates. If 30 items have the same issue date (say 2002), then those 30 items all appear in the index adjacent to each other, as opposed to a single 2002 entry.

+ +

Since dates in DSpace Dublin Core are in ISO8601, all in the UTC time zone, a simple alphanumeric sort is sufficient to sort by date, including dealing with varying granularities of date reasonably. For example:

+ + 
2001-12-10
+2002
+2002-04
+2002-04-05
+2002-04-09T15:34:12Z
+2002-04-09T19:21:12Z
+2002-04-10
+ +
Date Accessioned
+ +

In order to determine which items most recently appeared, rather than using the date of issue, an item's accession date is used. This is the Dublin Core field date.accessioned. In other aspects this index is identical to the date of issue index.

+ + +
Items by a Particular Author
+ +

One last operation the browse API can perform is to extract items by a particular author. They do not have to be primary author of an item for that item to be extracted. You can specify a scope, too; that is, you can ask for items by author X in collection Y, for example.

+ +

This particular flavour of browse is slightly simpler than the others. You cannot presently specify a particular subset of results to be returned. The API call will simply return all of the items by a particular author within a certain scope.

+ +

Note that the author of the item must exactly match the author passed in to the API; see the explanation about the caveats of the author index browsing to see why this is the case.

+ +
+ +

The API is generally invoked by creating a BrowseScope object, and setting the parameters for which particular part of an index you want to extract. This is then passed to the relevent Browse method call, which returns a BrowseInfo object which contains the results of the operation. The parameters set in the BrowseScope object are:

+ + + +

To illustrate, here is an example:

+ + + +

The results of invoking Browse.getItemsByTitle with the above parameters might look like this:

+ + 
        Rabble-Rousing Rabbis From Sardinia
+        Reality TV: Love It or Hate It?
+FOCUS>  The Really Exciting Research Video
+        Recreational Housework Addicts: Please Visit My House
+        Regional Television Variation Studies
+        Revenue Streams
+        Ridiculous Example Titles:  I'm Out of Ideas
+ +

Note that in the case of title and date browses, Item objects are returned as opposed to actual titles. In these cases, you can specify the 'focus' to be a specific item, or a partial or full literal value. In the case of a literal value, if no entry in the index matches exactly, the closest match is used as the focus. It's quite reasonable to specify a focus of a single letter, for example.

+ +

Being able to specify a specific item to start at is particularly important with dates, since many items may have the save issue date. Say 30 items in a collection have the issue date 2002. To be able to page through the index 20 items at a time, you need to be able to specify exactly which item's 2002 is the focus of the browse, otherwise each time you invoked the browse code, the results would start at the first item with the issue date 2002.

+ +

Author browses return String objects with the actual author names. You can only specify the focus as a full or partial literal String.

+ +

Another important point to note is that presently, the browse indices contain metadata for all items in the main archive, regardless of authorization policies. This means that all items in the archive will appear to all users when browsing. Of course, should the user attempt to access a non-public item, the usual authorization mechanism will apply. Whether this approach is ideal is under review; implementing the browse API such that the results retrieved reflect a user's level of authorization may be possible, but rather tricky.

+ + +

Index Maintenance

+ +

The browse API contains calls to add and remove items from the index, and to regenerate the indices from scratch. In general the content management API invokes the necessary browse API calls to keep the browse indices in sync with what is in the archive, so most applications will not need to invoke those methods.

+ +

If the browse index becomes inconsistent for some reason, the InitializeBrowse class is a command line tool (generally invoked using the /dspace/bin/index-all shell script) that causes the indices to be regenerated from scratch.

+ + +

Caveats

+ +

Presently, the browse API is not tremendously efficient. 'Indexing' takes the form of simply extracting the relevant Dublin Core value, normalising it (lower-casing and removing any leading article in the case of titles), and inserting that normalized value with the corresponding item ID in the appropriate browse database table. Database views of this table include collection and community IDs for browse operations with a limited scope. When a browse operation is performed, a simple SELECT query is performed, along the lines of:

+ + 
SELECT item_id FROM ItemsByTitle ORDER BY sort_title OFFSET 40 LIMIT 20
+ +

There are two main drawbacks to this: Firstly, LIMIT and OFFSET are PostgreSQL-specific keywords. Secondly, the database is still actually performing dynamic sorting of the titles, so the browse code as it stands will not scale particularly well. The code does cache BrowseInfo objects, so that common browse operations are performed quickly, but this is not an ideal solution.

+ + +

History Recorder

+ +

The purpose of the history subsystem is to capture a time-based record of significant changes in DSpace, in a manner suitable for later refactoring or repurposing. Note that the history data is not expected to provide current information about the archive; it simply records what has happened in the past.

+ +

The Harmony project describes a simple and powerful approach for modeling temporal data. The DSpace history framework adopts this model. The Harmony model is used by the serialization mechanism (and ultimately by agents who interpret the serializations); users of the History API need not be aware of it. The content management API handles invocations of the history system. Users of the DSpace public API do not generally need to use the history API.

+ +

When anything of archival interest occurs in DSpace, the saveHistory method of the HistoryManager is invoked. The parameters contains a reference to anything of archival interest. Upon reception of the object, it serializes the state of all archive objects referred to by it, and creates Harmony-style objects and associations to describe the relationships between the objects. (A simple example is given below). Note that each archive object must have a unique identifier to allow linkage between discrete events; this is discussed under "Unique IDs" below.

+ +

The serializations (including the Harmony objects and associations) are persisted as files in the /dspace/history (or other configured) directory. The history and historystate tables contain simple indicies into the serializations in the file system.

+ +

Archival Events

+ +

The following events are significant enough to warrant history records:

+ + + + +

Serializations

+ +

The serialization of an archival object consists of:

+ + + + +

Unique Ids

+ +

To be able to trace the history of an object, it is essential that the object have a unique identifier. Since not all objects in the system have Handles, the unique identifiers are only weakly tied to the Handle system. Instead, the identifier consists of:

+ + + + +

Storage

+ +

When an archive object is serialized, an object ID and MD5 checksum are recorded. When another object is serialized, the checksum for the serialization is matched against existing checksums for that object. If the checksum already exists, the object is not stored; a reference to the object is used instead. Note that since none of the serializations are deleted, reference counting is unnecessary.

+ +

The history data is not initially stored in a queryable form. Two simple RDBMS tables give basic indications of what is stored, and where. The history table is an index of serializations with checksums and dates. The history_id column corresponds to the file in which a serialization is stored. For example, if the history ID is 123456, it will be stored in the file:

+ + 
/dspace/history/00/12/34/123456
+ +

The table also contains the date the serialization was written and the MD5 checksum of the serialization.

+ +

The historystate table is supposed to indicate the most recent serialization of any given object.

+ + +

Example

+ +

An item is submitted to a collection via bulk upload. When (and if) the item is eventually added to the collection, the history method is called, with references to the item, its collection, the e-person who performed the bulk upload, and some indication of the fact that it was submitted via a bulk upload.

+ +

When called, the HistoryManager does the following: It creates the following new resources (all with unique ids):

+ + + +

It also generates the following relationships:

+ + 
event  --atTime-->     time
+event  --hasOutput-->  state
+Item   --inState-->    state
+state  --contains-->   Item
+action --creates-->    Item
+event  --hasAction-->  action
+action --usesTool-->   DSpace Upload
+action --hasAgent-->   User
+ +

The history component serializes the state of all archival objects involved (in this case, the item, the e-person, and the collection). It creates entries in the history database tables which associate the archival objects with the generated serializations.

+ +

Caveats

+ +

This history system is a largely untested experiment. It also needs further documentation. There have been no serious efforts to determine whether the information written by the history system, either to files or the database tables, is accurate. In particular, the historystate table does not seem to be correctly written.

+ +
+ +
+ Copyright © 2002-2004 MIT and Hewlett Packard +
+ + diff --git a/dspace/docs/configure.html b/dspace/docs/configure.html new file mode 100644 index 0000000000..fd0f0d9309 --- /dev/null +++ b/dspace/docs/configure.html @@ -0,0 +1,847 @@ + + + + DSpace System Documentation: Configuration and Customization + + + + +

DSpace System Documentation: Configuration and Customization

+

Back to contents

+

There are a number of ways in which DSpace can be configured and/or +customized:

+ +

Of these methods, only the last is likely to cause any headaches; if +you update the DSpace source code directly, it may make applying future +updates difficult. However, DSpace is open source, of course, and if +you make any modifications that might be helpful to other institutions +or organizations, feel free to send them to the DSpace team at MIT.

+

The dspace.cfg Configuration +Properties File

+

The primary way of configuring DSpace is to edit the dspace.cfg. +You'll definitely have to do this before you can operate DSpace +properly. dspace.cfg contains basic information about a +DSpace installation, including system path information, network host +information, and other things like site name.

+

The default dspace.cfg is a good source of +information, and contains comments for all properties. It's a basic +Java properties file, where lines are either comments, starting with a '#', +blank lines, or property/value pairs of the form:

+
property.name = property value
+

Due to time constraints, this document does not contain an +exhaustive list of properties; they are all listed in the supplied dspace.cfg. +Below are some particularly relevant properties with notes for their +use:

+ + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + +
dspace.cfg Main Properties (Not Complete)
PropertyExample ValuesNotes
dspace.dir/dspaceRoot directory of DSpace installation. Omit the trailing '/'. +Note that if you change this, there are several other parameters you +will probably want to change to match, e.g. assetstore.dir.
dspace.urlhttp://dspace.myu.edu
+ http://dspacetest.myu.edu:8080		Main URL at which DSpace Web UI webapp is deployed. Include +any port number, but do not include a trailing '/'
dspace.hostnamedspace.myu.eduFully qualified hostname; do not include port number
dspace.nameDSpace at My UniversityShort and sweet site name, used throughout Web UI, e-mails +and elsewhere (such as OAI protocol)
config.template.foo/opt/othertool/cfg/fooWhen install-configs is run, the file [dspace]/config/templates/foo +file will be filled out with values from dspace.cfg and +copied to the value of this property, in this example /opt/othertool/cfg/foo. + See here for more information.
webui.site.authenticatoredu.myu.MyAuthenticatorThe Java class name of a class implementing the org.dspace.app.webui.SiteAuthenticator +interface.
handle.prefix1721.1234The Handle prefix for your site, see the Handle section
assetstore.dir/bigdisk/storeThe location in the file system for asset (bitstream) store +number zero. This should be a directory for the sole use of DSpace.
assetstore.dir.n/anotherdisk/store1The location in the file system of asset (bitstream) store +number n. When adding additional stores, start with 1 (assetstore.dir.1 +and count upwards. Always leave asset store zero (assetstore.dir). +For more details, see the Bitstream +Storage section.
assetstore.incoming1The asset store number to use for storing new bitstreams. For +example, if assetstore.dir.1 is /anotherdisk/store1, +and assetstore.incoming is 1, new +bitstreams will be stored under /anotherdisk/store1. A +value of 0 (zero) corresponds to assetstore.dir. +For more details, see the Bitstream +Storage section.
srb.xxx
+ srb.xxx.n
+ 		/zone/home/user.domain
+ 		The sets of SRB access parameters (see dspace.cfg) if one or more SRB +accounts are used. The srb.xxx +set would correspond to asset (bitstream) store number zero. The srb.xxx.n set would correspond +to asset (bitstream) store number n. +For more details, see the Bitstream +Storage section.
webui.submit.enable-cctrueEnable the Creative Commons license step in the submission +process. Submitters are given an opportunity to select a Creative +Commons license to accompany the Item. Creative Commons licenses govern +the use of the content. For more details, see the Creative Commons website.
+

Whenever you edit dspace.cfg, you should then run [dspace]/bin/install-configs +so that any changes you may have made are reflected in the +configuration files of other applications, for example Apache. You may +then need to restart those applications, depending on what you changed.

+

Wording of E-mail Messages

+

Sometimes DSpace automatically sends e-mail messages to users, for +example to inform them of a new workflow task, or as a subscription +e-mail alert. The wording of emails can be changed by editing the +relevant file in [dspace]/config/emails. Each file +is commented. Be careful to keep the right number 'placeholders' (e.g.{2}).

+

Local DSpace Administrator Contact Information

+

There are several places in DSpace in which the user will be shown +contact information for the local DSpace Administrator: for instance, +when an error occurs, or in the on-line help when the user is looking +for more information. The contact information is displayed by [dspace-source]/jsp/components/contact-info.jsp. +This JSP retrieves the help e-mail in dspace.cfg, but the phone number +in the JSP is a dummy phone number that needs to be edited directly in +the JSP. You should be sure to edit this file (adding any additional +information you feel might be useful) so that users know who to contact +for further information.

+

The Dublin Core and Bitstream Format Registries

+

The [dspace]/config/registries directory +contains two XML files. These are used to load the initial +contents of the Dublin Core type registry and Bitstream Format +registry. After the initial loading (performed by ant +fresh_install above), the registries reside in the database; the +XML files are not updated.

+

Currently, the system requires that every item have a Dublin Core +record. The exact Dublin Core elements and qualifiers that are used can +be configured by editing the Dublin Core registry. This can either be +done at install-time, by editing [dspace]/config/registries/dublin-core-types.xml, +or at run-time using the administration Web UI. However, note that some +elements and qualifiers must be present for DSpace to function +correctly since they are used for various purposes by the code. Details +are in the relevant .xml file.

+

Also note that altering the Dublin Core registry does not, at the +current time, cause corresponding changes in the Web UI (e.g. the +submission interface or search indices).

+

The bitstream formats recognized by the system and levels of support +are similarly stored in the bitstream format registry. This can also be +edited at install-time via [dspace]/config/registries/bitstream-formats.xml +or by the administation Web UI. The contents of the bitstream format +registry are entirely up to you, though the system requires that the +following two formats are present:

+ +

Configuration Files for Other Applications

+

To ease the hassle of keeping configuration files for other +applications involved in running a DSpace site, for example Apache, in +sync, the DSpace system can automatically update them for you when the +main DSpace configuration is changed. This feature of the DSpace system +is entirely optional, but we found it useful.

+

The way this is done is by placing the configuration files for those +applications in [dspace]/config/templates, and +inserting special values in the configuration file that will be filled +out with appropriate DSpace configuration properties. Then, tell DSpace +where to put filled-out, 'live' version of the configuration by adding +an appropriate property to dspace.cfg, and run [dspace]/bin/install-configs.

+

Take the apache13.conf file as an example. This +contains plenty of Apache-specific stuff, but where it uses a value +that should be kept in sync across DSpace and associated applications, +a 'placeholder' value is written. For example, the host name:

+
ServerName @@dspace.hostname@@
+

The text @@dspace.hostname@@ will be filled out with +the value of the dspace.hostname property in dspace.cfg. +Then we decide where we want the 'live' version, that is, the version +actually read in by Apache when it starts up, will go.

+

Let's say we want the live version to be located at /opt/apache/conf/dspace-httpd.conf. +To do this, we add the following property to dspace.cfg +so DSpace knows where to put it:

+
config.template.apache13.conf = /opt/apache/conf/dspace-httpd.conf
+

Now, we run [dspace]/bin/install-configs. This +reads in [dspace]/config/templates/apache13.conf, +and places a copy at /opt/apache/conf/dspace-httpd.conf +with the placeholders filled out.

+

So, in /opt/apache/conf/dspace-httpd.conf, there will +be a line like:

+
ServerName dspace.myu.edu
+

The advantage of this approach is that if a property like the +hostname changes, you can just change it in dspace.cfg +and run install-configs, and all of your tools' +configuration files will be updated.

+

However, take care to make all your edits to the versions in [dspace]/config/templates! +It's a wise idea to put a big reminder at the top of each file, since +someone might unwittingly edit a 'live' configuration file which would +later be overwritten.

+

Customizing the Web User Interface

+

The Web UI is implemented using Java Servlets which handle the +business logic, and JavaServer Pages (JSPs) which produce the HTML +pages sent to an end-user. Since the JSPs are much closer to HTML than +Java code, altering the look and feel of DSpace is relatively easy.

+

To make it even easier, DSpace allows you to 'override' the JSPs +included in the source distribution with modified versions, that are +stored in a separate place, so when it comes to updating your site with +a new DSpace release, your modified versions will not be overwritten.

+

However, note that the data (attributes) passed from an underlying +Servlet to the JSP may change between versions, so you may have to +modify your customized Servlet to deal with the new data.

+

The JSPs are stored in [dspace-source]/jsp. +Place your edited version of a JSP in the [dspace-source]/jsp/local +directory, with the same path as the original. If they exist, these +will be used in preference to the distributed versions in [dspace-source]/jsp. +For example:

+ + + + + + + + + + + + + + + +
DSpace defaultLocally-modified version
[dspace-source]/jsp/community-list.jsp[dspace-source]/jsp/local/community-list.jsp
[dspace-source]/jsp/mydspace/main.jsp[dspace-source]/jsp/local/mydspace/main.jsp
+

Heavy use is made of a style sheet, in [dspace-source]/jsp/styles.css.jsp. +If you make edits, call the local version [dspace-source]/jsp/local/styles.css.jsp, +and it will be used automatically in preference to the default, as +described above.

+

Fonts and colors can be easily changed using the stylesheet. The +stylesheet is a JSP so that the user's browser version can be detected +and the stylesheet tweaked accordingly.

+

The 'layout' of each page, that is, the top and bottom banners and +the navigation bar, are determined by the JSPs [dspace-source]/jsp/layout/header-*.jsp +and [dspace-source]/jsp/layout/footer-*.jsp. You +can provide modified versions of these (in [dspace-source]/jsp/local/layout, +or define more styles and apply them to pages by using the "style" +attribute of the dspace:layout tag.

+

After you've customized your JSPs, you must rebuild the +DSpace Web application. If you haven't already built and +installed it, follow the install +directions. Otherwise, follow the steps below:

+
    +
  1. +

    Rebuild the dspace.war file by running the +following command from your [dspace-source] +directory:

    + 
    ant -Dconfig=[dspace]/config/dspace.cfg build_wars
    +
    2. +
  2. +

    Shut down Tomcat, and delete the existing [tomcat]/webapps/dspace +directory.

    +
    3. +
  3. +

    Copy the new .war file to the Tomcat webapps directory:

    + 
    cp [dspace-source]/build/dspace.war [tomcat]/webapps
    +
    4. +
+

When you restart the web server you should see your customized JSPs.

+

Custom Authentication Code

+

Since many institutions and organizations have exisiting +authentication systems, DSpace has been designed to allow these to be +easily integrated. To do this, you can provide a custom class +implementing the Java interface org.dspace.app.webui.SiteAuthenticator. +These methods are invoked when various authentication-related events +occur in the Web user interface.

+

The basic authentication procedure in the DSpace Web UI is this:

+
    +
  1. A request is received from an end-user's browser that, if +fulfilled, would lead to an action requiring authorization taking place.
    2. +
  2. If the end-user is already authenticated: + +
    3. +
+

LDAP Authentication

+

If LDAP is enabled in the dspace.cfg file, then new users +will be able to register by entering their username and +password without being sent the registration token. If +users do not have a username and password, then they +can still register and login with just their email address +the same way they do now. +

+

If you want to give any special privileges to LDAP users, +you will still need to extend the SiteAuthenticator class to +automatically put people who have a netid into a special +group. You might also want to give certain email addresses +special privileges. Refer to the Custom +Authentication Code section above for more information about +how to do this. +

+

Here is an explanation of what each of the different +configuration parameters are for: +

+

+

System Statistical Reports

+ +

Statistics for the system can be made available at http://www.mydspaceinstance.edu/statistics. To use the system statistics you will have to initialise them as per the installation documentation, but before you do so you need to perform the customisations discussed here in order to ensure that the reports are generated correctly.

+ +

Configuration File

+ +

Configuration for the statistics system are in [dspace]/config/dstat.cfg and the file should guide you to correctly filling in the details required. For the most part you will not need to change this file.

+ +

Customising Shell Scripts

+ +

+To customise the supplied perl scripts to do monthly and general report +generation it is necessary to modify the scripts themselves sightly. This +is because these scripts were developed to speed up the process of using +DStat at Edinburgh University Library and were not particularly intended for +external use. They appear here for the convenience of others and in order to +bridge the gap between the report generation and the inclusion of those reports +into the DSpace UI, which is currently a clunky process. +

+ +

+In order to get these scripts to work for you, open each of the following in +turn: +

+ + 
+dstat-general
+dstat-initial
+dstat-monthly
+dstat-report-general
+dstat-report-initial
+dstat-report-monthly
+
+ +

+scripts eding with -general do the work for building reports spanning the +entire history of the archive; scripts ending -initial are to initialise the +reports by doing monthly reports from some start date up to the present; +scripts ending -monthly generate a single monthly report for the current +month. These scripts are just designed to make life easier, and are not +particularly clever or elegant. +

+ +

+In each file you will find a section: +

+ + 
+# Details used
+################################################
+
+... some perl ...
+
+################################################
+
+ +

+the perl between the lines of hashes defines the variables which will be used +to do all of the processing in the report. The following explains what the +variables mean and what they should be set to for each of the scripts +

+ +

+dstat-initial:
+ +$out_prefix: prefix to place in front of each output file.
+$out_suffix: suffix for output file. A date will be inserted between the + prefix and suffix
+$start_year: year to start back-analysing monthly logs from
+$start_month: month to start back-analysing monthly logs from
+$dsrun: path to your dsrun script, usually [dspace]/bin/dsrun
+$out_directory: directory into which to place analysis files, for example + [dspace]/bin/log/
+

+ +

+dstat-monthly:
+ +$out_prefix: prefix to place in front of each output file.
+$out_suffix: suffix for output file. A date will be inserted between the + prefix and suffix
+$dsrun: path to your dsrun script, usually [dspace]/bin/dsrun
+$out_directory: directory into which to place analysis files, for example + [dspace]/bin/log/
+

+ +

+dstat-general:
+ +$out_prefix: prefix to place in front of each output file.
+$out_suffix: suffix for output file. Today's date will be inserted between the + prefix and suffix
+$dsrun: path to your dsrun script, usually [dspace]/bin/dsrun
+$out_directory: directory into which to place analysis files, for example + [dspace]/bin/log/
+

+ +

+dstat-report-initial:
+ +$in_prefix: the prefix of the files generated by dstat-initial
+$in_suffix: the suffix of the files generated by dstat-initial
+$out_prefix: the report file prefix. Should be "report-" in order to work with + DSpace UI
+$out_suffix: the report file suffix. Should be ".html" in order to work with + DSpace UI
+$start_year: the start year used in dstat-initial
+$start_month: the start month used in dstat-initial
+$dsrun: path to your dsrun script, usually [dspace]/bin/dsrun
+$in_directory: directory where analysis files were placed in dstat-initial
+$out_directory: the live reports directory: [dspace]/reports/
+

+ +

+dstat-report-monthly:
+ +$in_prefix: the prefix of the files generated by dstat-monthly
+$in_suffix: the suffix of the files generated by dstat-monthly
+$out_prefix: the report file prefix. Should be "report-" in order to work with + DSpace UI
+$out_suffix: the report file suffix. Should be ".html" in order to work with + DSpace UI
+$dsrun: path to your dsrun script, usually [dspace]/bin/dsrun
+$in_directory: directory where analysis files were placed in dstat-monthly
+$out_directory: the live reports directory: [dspace]/reports/
+

+ +

+dstat-report-general:
+ +$in_prefix: the prefix of the files generated by dstat-general
+$in_suffix: the suffix of the files generated by dstat-general
+$out_prefix: the report file prefix. Should be "report-general-" in order to + work with DSpace UI
+$out_suffix: the report file suffix. Should be ".html" in order to work with + DSpace UI
+$dsrun: path to your dsrun script, usually [dspace]/bin/dsrun
+$in_directory: directory where analysis files were placed in dstat-general
+$out_directory: the live reports directory: [dspace]/reports/
+

+ +

+If you want additional customisations, you will need to modify the lines which +build the command to be executed and change the parameters passed to the java +processes which actually carry out the analysis. For more information on these +processes either build the javadocs or run: +

+ + +[dspace]/bin/dsrun ac.ed.dspace.stats.LogAnalyser -help
+ +[dspace]/bin/dsrun ac.ed.dspace.stats.ReportGenerator -help + +

Cron Jobs

+ +

In order that the reports should be generated regularly and thus kept up to date you should set up the following cron jobs:

+ + 
+#run stat analyses
+0 1 * * * [dspace]/bin/stat-general
+0 1 * * * [dspace]/bin/stat-monthly
+0 2 * * * [dspace]/bin/stat-report-general
+0 2 * * * [dspace]/bin/stat-report-monthly
+
+ +

Obviously, you should choose execution times which are most useful to you, and you should ensure that the -report- scripts run a short while after the analysis scripts to give them time to complete (a run of around 8 months worth of logs can take around 25 seconds to complete); the resulting reports will let you know how long analysis took and you can adjust your cron times accordingly.

+ + + +
  • If the end-user is NOT authenticated, i.e. is accessing DSpace +anonymously: + +
    • + +

    Please see the SiteAuthenticator.java source file for +information about each of the methods. The default implementation, org.dspace.app.webui.SimpleAuthenticator, +is a simple implementation that implements these policies:

    + +

    Included in the source is the implementation of SiteAuthenticator +used at MIT, edu.mit.dspace.MITAuthenticator. This +implements a slightly more complex authentication mechanism:

    + +

    The X509 certificate login servlet has an extra feature: If the webui.cert.autoregister +configuration property is true, it will automatically +register the user with the system.

    +

    You could create a customized version of the password login servlet +to perform a similar action. For example, if your organization uses +Windows NT domain authentication, you could implement a version of +PasswordServlet.java that validates against Windows NT authentication, +and automatically adds an e-person record for new users. It is strongly +recommended that you do not edit PasswordServlet but create a new +servlet for this, so that future updates of the DSpace code do not +overwrite your changes. You would also have to implement a customized SiteAuthenticator +in which the startAuthentication method would forward +requests to your new servlet.

    +

    Displaying Image Thumbnails

    +

    Browse and Search Results Page Thumbnails

    +

    Image thumbnails can be enabled on the Browse and Search Results +pages by setting the appropriate configuration values. To enable the +display of thumbnails the following items must be set in the dspace.cfg +file:

    +

    webui.browse.thumbnail.show = true

    +

    If set to false or this configuration item is missing +then thumbnails will not be shown.

    +

    The size of the browse/search thumbnails can also be configured to a +smaller size than that generated by the mediafilter. To do this set the +following configuration items:

    +

    webui.browse.thumbnail.maxheight = <maxheight in pixels>

    +

    webui.browse.thumbnail.maxwidth = <maxwidth in pixels>

    +

    If these configuration items are not set, thumbnail.maxheight +and thumbnail.maxwidth are used. Setting these values +greater than or equal to the size of the thumbnail generated by the +mediafilter (i.e. thumbnail.maxheight and thumbnail.maxwidth) +will have no effect.

    +

    Note:

    + +

    Configuring Thumbnail Link Behaviour

    +

    The target of a thumbnail in the Browse and Search Results Page can +be configured by setting the following configuration item:

    +

    webui.browse.thumbnail.linkbehaviour = <target page type>

    +

    Currently the values item and bitstream +are allowed. If this configuration item is not set, or set incorrectly, +the default is item.

    +

    Item Display Page Thumbnails

    +

    Thumbnails may also be enabled or disabled on the Item Display page +by setting the following configuration item in dspace.cfg:

    +

    webui.item.thumbnail.show = true

    +

    If set to false or this configuration item is missing +then thumbnails will not be shown.

    +

    Displaying Community and Collection Item Counts

    +

    To show the item count against communities and collections set the webui.strengths.show +configuration item in the dspace.cfg file as follows:

    +

    webui.strengths.show = true

    +

    If this config item is missing or is set to any value other than true +the item counts will not be shown

    + +

    Internationalisation

    +

    +The Java Standard Tag Library v1.0 is used to specify messages in the JSPs like this: + +

    +

    +OLD: +

    <H1>Search Results</H1>
+
    +

    +

    +NEW: +

    <H1><fmt:message key="jsp.search.results.title" /></H1>
+
    +

    + +

    +This message can now be changed using the config/Messages_en.properties file. (This must be done at build-time: Messages_en.properties is placed in the dspace.war Web application file.) +

    + +
    jsp.search.results.title = Search Results
+

    +Phrases may have parameters to be passed in, to make the job of translating easier, reduce the number of 'keys' and to allow translators to make the translated text flow more appropriately for the target language. +

    + +

    +OLD: +

    <P>Results <%= r.getFirst() %> to <%= r.getLast() %> of <%= r.getTotal() %></P>
+
    +

    +

    +NEW: + +

    <fmt:message key="jsp.search.results.text">
+  <fmt:param><%= r.getFirst() %></fmt:param>
+  <fmt:param><%= r.getLast() %></fmt:param>
+  <fmt:param><%= r.getTotal() %></fmt:param>
+
+</fmt:message>
+
    +

    +

    +(Note: JSTL 1.0 does not seem to allow JSP <%= %> expressions to be passed in as values of attribute in <fmt:param value=""/>) +

    +

    +The above would appear in the Messages_xx.properties file as: + +

    + +
    jsp.search.results.text = Results {0}-{1} of {2}
+

    +Introducing number parameters that should be formatted according to the locale used makes no difference in the message key compared to atring parameters: +

    + +
    jsp.submit.show-uploaded-file.size-in-bytes = {0} bytes
+

    +In the JSP using this key can be used in the way belov: +

    + +
    <fmt:message key="jsp.submit.show-uploaded-file.size-in-bytes">
+  <fmt:param><fmt:formatNumber><%= bitstream.getSize() %></fmt:formatNumber></fmt:param>
+
+</fmt:message>
+

    +(Note: JSTL offers a way to include numbers in the message keys as jsp.foo.key = {0,number} bytes. Setting the parameter as <fmt:param value="${variable}" /> workes when variable is a single variable name and doesn't work when trying to use a method's return value instead: bitstream.getSize(). Passing the number as string (or using the <%= %> expression) also does not work.) + +

    +

    +Multiple Messages.properties can be created for different languages. See ResourceBundle.getBundle. e.g. you can add German and Canadian French translations: +

    + +
    Messages_de.properties
+Messages_fr_CA.properties
+

    +The end user's browser settings determine which language is used. Messages_en.properties (or the default server locale) will be used as a default if there's no language bundle for the end user's preferred language. +

    + +

    +The dspace:layout tag has been updated to allow dictionary keys to be passed in for the titles. It now has two new parameters: titlekey and parenttitlekey. So where before you'd do: +

    + +
    <dspace:layout title="Here"
+               parentlink="/mydspace"
+               parenttitle="My DSpace">
+

    +You now do: +

    <dspace:layout titlekey="jsp.page.title"
+               parentlink="/mydspace"
+               parenttitlekey="jsp.mydspace">
+
+
    And so the layout tag itself gets the relavent stuff out of the dictionary. title and parenttitle still work as before for backwards compatibility, and the odd spot where that's preferable. +

    +

    Message Key Convention

    +

    +When translating further pages, please follow the convention for naming message keys to avoid clashes. +

    +

    +For text in JSPs use the complete path + filename of the JSP, then a one-word name for the message. e.g. for the title of jsp/mydspace/main.jsp use: + +

    jsp.mydspace.main.title
+
    +

    +

    +Some common words (e.g. "Help") can be brought out into keys starting jsp. for ease of translation, e.g.: +

    jsp.admin = Administer
+
    +

    +

    +Other common words/phrases are brought out into 'general' parameters if they relate to a set (directory) of JSPs, e.g. +

    + +
    jsp.tools.general.delete = Delete
+

    + +Phrases that relate strongly to a topic (eg. MyDSpace) but used in many JSPs outside the particular directory are more convenient to be cross-referenced. For example one could use the key below in jsp/submit/saved.jsp to provide a link back to the user's MyDSpace: +

    +

    +(Cross-referencing of keys in general is not a good idea as it may make maintenance more difficult. But in some cases it has more advantages as the meaning is obvious.) +

    jsp.mydspace.general.goto-mydspace = Go to My DSpace
+
+
    +

    +

    +For text in servlet code, in custom JSP tags or wherever applicable use the fully qualified classname + a one-word name for the message. e.g. +

    org.dspace.app.webui.jsptag.ItemListTag.title = Title
+org.dspace.app.webui.servlet.CommunityListServlet.
+
    +

    +

    Which Languages are currently supported?

    +

    To view translations currently being developed, please refer to the i18n page of the DSpace Wiki. + +

    On-line Help About File Formats

    +

    Because the file format support policy is determined by each +individual institution, the on-line help on this subject is +intentionally left blank. The help file will, however, retrieve a list +of formats and the support levels associated with them in your database +and display this information to the user. We highly recommend that you +edit the "Format Support Policy" section of the file [dspace-source]/jsp/help/formats.jsp.

    +
    +
    Copyright © 2002-2004 MIT and Hewlett Packard
    + + diff --git a/dspace/docs/directories.html b/dspace/docs/directories.html new file mode 100644 index 0000000000..c538e31183 --- /dev/null +++ b/dspace/docs/directories.html @@ -0,0 +1,135 @@ + + + + DSpace System Documentation: Directories and Files + + + + +

    DSpace System Documentation: Directories and Files

    + +

    Back to contents

    + +

    Overview

    + +

    A complete DSpace installation consists of three separate directory trees: + +

    +
    The source directory:
    +

    This is where (surprise!) the source code lives. Note that the config files here are used only during the initial install process. After the install, config files should be changed in the install directory. It is referred to in this document as [dspace-source].

    + +
    The install directory:
    +

    This directory is populated during the install process and also by DSpace as it runs. It contains config files, command-line tools (and the libraries necessary to run them), and usually--although not necessarily--the contents of the DSpace archive (depending on how DSpace is configured). After the initial build and install, changes to config files should be made in this directory. It is referred to in this document as [dspace].

    + +
    The web deployment directory:
    +

    This directory is generated by the web server the first time it finds a dspace.war file in its webapps directory. It contains the unpacked contents of dspace.war, i.e. the JSPs and java classes and libraries necessary to run DSpace. Files in this directory should never be edited directly; if you wish to modify your DSpace installation, you should edit files in the source directory and then rebuild. The contents of this directory aren't listed here since its creation is completely automatic. It is usually referred to in this document as [tomcat]/webapps/dspace.

    +
    + +

    Source Directory Layout

    + + + + +

    Installed Directory Layout

    + +

    Below is the basic layout of a DSpace installation using the default configuration. These paths can be configured if necessary.

    + + + + +

    Log Files

    + +

    The first source of potential confusion is the log files. Since DSpace uses a number of third-party tools, problems can occur in a variety of places. Below is a table listing the main log files used in a typical DSpace setup. The locations given are defaults, and might be different for your system depending on where you installed DSpace and the third-party tools. The ordering of the list is roughly the recommended order for searching them for the details about a particular problem or error.

    + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + +
    DSpace Log File Locations
    Log FileWhat's In It
    [dspace]/log/dspace.logMain DSpace log file. This is where the DSpace code writes a simple log of events and errors that occur within the DSpace code. You can control the verbosity of this by editing the [dspace]/config/templates/log4j.properties file and then running [dspace]/bin/install-configs.
    [tomcat]/logs/catalina.outThis is where Tomcat's standard output is written. Many errors that occur within the Tomcat code are logged here. For example, if Tomcat can't find the DSpace code (dspace.jar), it would be logged in catalina.out.
    [tomcat]/logs/hostname_log.yyyy-mm-dd.txtIf you're running Tomcat stand-alone (without Apache), it logs some information and errors for specific Web applications to this log file. hostname will be your host name (e.g. dspace.myu.edu) and yyyy-mm-dd will be the date.
    [tomcat]/logs/apache_log.yyyy-mm-dd.txtIf you're using Apache, Tomcat logs information about Web applications running through Apache (mod_webapp) in this log file (yyyy-mm-dd being the date.)
    [apache]/error_logApache logs to this file. If there is a problem with getting mod_webapp working, this is a good place to look for clues. Apache also writes to several other log files, though error_log tends to contain the most useful information for tracking down problems.
    [dspace]/log/handle-plug.logThe Handle server runs as a separate process from the DSpace Web UI (which runs under Tomcat's JVM). Due to a limitation of log4j's 'rolling file appenders', the DSpace code running in the Handle server's JVM must use a separate log file. The DSpace code that is run as part of a Handle resolution request writes log information to this file. You can control the verbosity of this by editing [dspace]/config/templates/log4j-handle-plugin.properties.
    [dspace]/log/handle-server.logThis is the log file for CNRI's Handle server code. If a problem occurs within the Handle server code, before DSpace's plug-in is invoked, this is where it may be logged.
    [dspace]/handle-server/error.logOn the other hand, a problem with CNRI's Handle server code might be logged here.
    PostgreSQL logPostgreSQL also writes a log file. This one doesn't seem to have a default location, you probably had to specify it yourself at some point during installation. In general, this log file rarely contains pertinent information--PostgreSQL is pretty stable, you're more likely to encounter problems with connecting via JDBC, and these problems will be logged in dspace.log.
    + +
    + +
    + Copyright © 2002-2004 MIT and Hewlett Packard +
    + + diff --git a/dspace/docs/functional.html b/dspace/docs/functional.html new file mode 100644 index 0000000000..5de2d05537 --- /dev/null +++ b/dspace/docs/functional.html @@ -0,0 +1,765 @@ + + + + DSpace System +Documentation: Functional Overview + + + + +

    DSpace +System Documentation: +Functional Overview

    +

    Back to contents

    +

    The following sections describe +the various functional aspects of the DSpace system.

    +

    Data Model

    +

    Data Model Diagram

    +

    Data +Model Diagram

    +

    The way data is organized in +DSpace is intended to reflect the structure of the organization using +the DSpace system. Each DSpace site is divided into communities; +these typically correspond to a laboratory, research center or +department. As of DSpace version 1.2, these communities can be +organized into an hierarchy.

    +

    Communities contain collections, +which are groupings of related content. A collection may appear in more +than one community.

    +

    Each collection is composed of items, +which are the basic archival elements of the archive. Each item is +owned by one collection. Additionally, an item may appear in additional +collections; however every item has one and only one owning collection.

    +

    Items are further subdivided +into named bundles +of bitstreams. +Bitstreams are, as the name suggests, streams of bits, usually ordinary +computer files. Bitstreams that are somehow closely related, for +example HTML files and images that compose a single HTML document, are +organised into bundles.

    +

    In practice, most items tend to +have three named bundles:

    + +

    Each bitstream is associated +with one Bitstream Format. +Because preservation services may be an important aspect of the DSpace +service, it is important to capture the specific formats of files that +users submit. In DSpace, a bitstream format is a unique and consistent +way to refer to a particular file format. An integral part of a +bitstream format is an either implicit or explicit notion of how +material in that format can be interpreted. For example, the +interpretation for bitstreams encoded in the JPEG standard for still +image compression is defined explicitly in the Standard ISO/IEC +10918-1. The interpretation of bitstreams in Microsoft Word 2000 format +is defined implicitly, through reference to the Microsoft Word 2000 +application. Bitstream formats can be more specific than MIME types or +file suffixes. For example, application/ms-word +and .doc +span multiple versions of the Microsoft Word application, each of which +produces bitstreams with presumably different characteristics.

    +

    Each bitstream format +additionally has a support +level, indicating how well the +hosting institution is likely to be able to preserve content in the +format in the future. There are three possible support levels that +bitstream formats may be assigned by the hosting institution. The host +institution should determine the exact meaning of each support level, +after careful consideration of costs and requirements. MIT Libraries' +interpretation is shown below:

    + + + + + + + + + + + + + + + +
    MIT Libraries' +Definitions of Bitstream Format Support Levels
    SupportedThe format is +recognized, and the hosting institution is confident it can make +bitstreams of this format useable in the future, using whatever +combination of techniques (such as migration, emulation, etc.) is +appropriate given the context of need.
    KnownThe format is +recognized, and the hosting institution will promise to preserve the +bitstream as-is, and allow it to be retrieved. The hosting institution +will attempt to obtain enough information to enable the format to be +upgraded to the 'supported' level.
    UnsupportedThe format is +unrecognized, but the hosting institution will undertake to preserve +the bitstream as-is and allow it to be retrieved.
    +

    Each item has one qualified +Dublin Core metadata record. Other metadata might be stored in an item +as a serialized bitstream, but we store Dublin Core for every item for +interoperability and ease of discovery. The Dublin Core may be entered +by end-users as they submit content, or it might be derived from other +metadata as part of an ingest process.

    +

    Items can be removed from DSpace in +one of two ways: They may be +'withdrawn', which means they remain in the archive but are completely +hidden from view. In this case, if an end-user attempts to access the +withdrawn item, they are presented with a 'tombstone,' that indicates +the item has been removed. For whatever reason, an item may also be +'expunged' if necessary, in which case all traces of it are removed +from the archive.

    + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + +
    Objects in the DSpace +Data Model
    ObjectExample
    CommunityLaboratory of Computer +Science; Oceanographic Research Center
    CollectionLCS Technical Reports; +ORC Statistical Data Sets
    ItemA technical report; a +data set with accompanying description; a video recording of a lecture
    BundleA group of HTML and +image bitstreams making up an HTML document
    BitstreamA single HTML file; a +single image file; a source code file
    Bitstream FormatMicrosoft Word version +6.0; JPEG encoded image format
    +

    Metadata

    +

    Broadly speaking, DSpace holds +three sorts of metadata about archived content:

    +
    +
    Descriptive Metadata
    +
    +

    Each Item +has one qualified Dublin Core metadata record. The set +of elements and qualifiers used by MIT Libraries +is the default configuration included in the DSpace source code. These +are loosely based on the Library +Application Profile set of +elements and qualifiers, though there are some differences.

    +

    Other descriptive metadata +about items may be held in serialized bitstreams. Communities +and collections +have some simple descriptive metadata (a name, and some descriptive +prose), held in the DBMS.

    +
    +
    Administrative Metadata
    +
    +

    This includes preservation +metadata, provenance and authorization policy data. Most of this is +held within DSpace's relation DBMS schema. Provenance metadata (prose) +is stored in Dublin Core records. Additionally, some other +administrative metadata (for example, bitstream byte sizes and MIME +types) is replicated in Dublin Core records so that it is easily +accessible outside of DSpace.

    +
    +
    Structural Metadata
    +
    +

    This includes information +about how to present an item, or bitstreams within an item, to an +end-user, and the relationships between constituent parts of the item. +As an example, consider a thesis consisting of a number of TIFF images, +each depicting a single page of the thesis. Structural metadata would +include the fact that each image is a single page, and the ordering of +the TIFF images/pages. Structural metadata in DSpace is currently +fairly basic; within an item, bitstreams can be arranged into separate +bundles as described +above. A bundle may also +optionally have a primary +bitstream. This is currently +used by the HTML +support to indicate which +bitstream in the bundle is the first HTML file to send to a browser.

    +

    In addition to some basic +technical metadata, bitstreams also have a 'sequence ID' that uniquely +identifies it within an item. This is used to produce a 'persistent' bitstream +identifier for each bitstream.

    +

    Additional structural +metadata can be stored in serialized bitstreams, but DSpace does not +currently understand this natively.

    +
    +
    +

    E-people

    +

    Many of DSpace's features such +as document discovery and retrieval can be used anonymously, but users +must be authenticated to perform functions such as submission, email +notification ('subscriptions') or administration. Users are also +grouped for easier administration. DSpace calls users e-people, +to reflect that some users may be machines rather than actual people.

    +

    DSpace hold the following +information about each e-person:

    + +

    E-people authenticate with +username/password pairs or X509 certificates. E-people can be members +of 'groups' to make administrator's lives easier when manipulating +authorization policies.

    +

    Authorization

    +

    DSpace's authorization system +is based on associating actions with objects and the lists of EPeople +who can perform them. The associations are called Resource Policies, +and the lists of EPeople are called Groups. There are two special +groups: 'administrators', who can do anything in a site, and +'anonymous', which is a list that contains all users. Assigning a +policy for an action on an object to anonymous means giving everyone +permission to do that action. (For example, most objects in DSpace +sites have a policy of 'anonymous' READ.) Permissions must be explicit +- lack of an explicit permission results in the default policy of +'deny'. Permissions also do not 'commute'; for example, if an e-person +has READ permission on an item, they might not necessarily have READ +permission on the bundles and bitstreams in that item. Currently +Collections, Communities and Items are discoverable in the browse and +search systems regardless of READ authorization.

    +

    The following actions are +possible:

    +

    Community

    + + + + + + + +
    ADD/REMOVEadd or remove +collections or sub-communities
    +

    Collection

    + + + + + + + + + + + + + + + + + + + +
    ADD/REMOVEadd or remove items (ADD += permission to submit items)
    DEFAULT_ITEM_READinherited as READ by all +submitted items
    DEFAULT_BITSTREAM_READinherited as READ by +bitstreams of all submitted items
    COLLECTION_ADMINcollection admins can +edit items in a collection, withdraw items, map other items into this +collection.
    +

    Item

    + + + + + + + + + + + + + + + +
    ADD/REMOVEadd or remove bundles
    READcan view item (item +metadata is always viewable)
    WRITEcan modify item
    +

    Bundle

    + + + + + + + +
    ADD/REMOVEadd or remove bitstreams +to a bundle
    +

    Bitstream

    + + + + + + + + + + + +
    READview bitstream
    WRITEmodify bitstream
    +

    Note that there is no 'DELETE' +action. In order to 'delete' an object (e.g. an item) from the archive, +one must have REMOVE permission on all objects (in this case, +collection) that contain it. The 'orphaned' item is automatically +deleted.

    +

    Policies can apply to +individual e-people or groups of e-people.

    +

    Ingest Process and Workflow

    +

    Rather than being a single +subsystem, ingesting is a process that spans several. Below is a simple +illustration of the current ingesting process in DSpace.

    +

    Ingest Process Diagram

    +

    DSpace +Ingest Process

    +

    The batch item importer is an +application, which turns an external SIP (an XML metadata document with +some content files) into an "in progress submission" object. The Web +submission UI is similarly used by an end-user to assemble an "in +progress submission" object.

    +

    Depending on the policy of the +collection to which the submission in targeted, a workflow process may +be started. This typically allows one or more human reviewers or +'gatekeepers' to check over the submission and ensure it is suitable +for inclusion in the collection.

    +

    When the Batch Ingester or Web +Submit UI completes the InProgressSubmission object, and invokes the +next stage of ingest (be that workflow or item installation), a +provenance message is added to the Dublin Core which includes the +filenames and checksums of the content of the submission. Likewise, +each time a workflow changes state (e.g. a reviewer accepts the +submission), a similar provenance statement is added. This allows us to +track how the item has changed since a user submitted it. (The History system +is also invoked, but provenance is easier for us to access at the +moment.)

    +

    Once any workflow process is +successfully and positively completed, the InProgressSubmission object +is consumed by an "item installer", that converts the +InProgressSubmission into a fully blown archived item in DSpace. The +item installer:

    + +

    Workflow Steps

    +

    A collection's workflow can +have up to three steps. Each collection may have an associated e-person +group for performing each step; if no group is associated with a +certain step, that step is skipped. If a collection has no e-person +groups associated with any step, submissions to that collection are +installed straight into the main archive.

    +

    In other words, the sequence is +this: The collection receives a submission. If the collection has a +group assigned for workflow step 1, that step is invoked, and the group +is notified. Otherwise, workflow step 1 is skipped. Likewise, workflow +steps 2 and 3 are performed if and only if the collection has a group +assigned to those steps.

    +

    When a step is invoked, the +task of performing that workflow step put in the 'task pool' of the +associated group. One member of that group takes the task from the +pool, and it is then removed from the task pool, to avoid the situation +where several people in the group may be performing the same task +without realizing it.

    +

    The member of the group who has +taken the task from the pool may then perform one of three actions:

    + + + + + + + + + + + + + + + + + + + +
    Workflow StepPossible actions
    1Can accept submission +for inclusion, or reject submission.
    2Can edit metadata +provided by the user with the submission, but cannot change the +submitted files. Can accept submission for inclusion, or reject +submission.
    3Can edit metadata +provided by the user with the submission, but cannot change the +submitted files. Must then commit to archive; may not reject submission.
    +

    Submission Workflow Diagram

    +

    Submission +Workflow in DSpace

    +

    If a submission is rejected, +the reason (entered by the workflow participant) is e-mailed to the +submitter, and it is returned to the submitter's 'My DSpace' page. The +submitter can then make any necessary modifications and re-submit, +whereupon the process starts again.

    +

    If a submission is 'accepted', +it is passed to the next step in the workflow. If there are no more +workflow steps with associated groups, the submission is installed in +the main archive.

    +

    One last possibility is that a +workflow can be 'aborted' by a DSpace site administrator. This is +accomplished using the administration UI.

    +

    The reason for this apparently +arbitrary design is that is was the simplist case that covered the +needs of the early adopter communities at MIT. The functionality of the +workflow system will no doubt be extended in the future.

    +

    Handles

    +

    Researchers require a stable +point of reference for their works. The simple evolution from sharing +of citations to emailing of URLs broke when Web users learned that +sites can disappear or be reconfigured without notice, and that their +bookmark files containing critical links to research results couldn't +be trusted long term. To help solve this problem, a core DSpace feature +is the creation of persistent identifier for every item, collection and +community stored in DSpace. To persist identifier, DSpace requires a +storage- and location- independent mechanism for creating and +maintaining identifiers. DSpace uses the CNRI Handle System +for creating these identifiers. The rest of this section assumes a +basic familiarity with the Handle system.

    +

    DSpace uses Handles primarily +as a means of assigning globally unique identifiers to objects. Each +site running DSpace needs to obtain a Handle 'prefix' from CNRI, so we +know that if we create identifiers with that prefix, they won't clash +with identifiers created elsewhere.

    +

    Presently, Handles are assigned +to communities, collections, and items. Bundles and bitstreams are not +assigned Handles, since over time, the way in which an item is encoded +as bits may change, in order to allow access with future technologies +and devices. Older versions may be moved to off-line storage as a new +standard becomes de facto. Since it's usually the item +that is being preserved, rather than the particular bit encoding, it +only makes sense to persistently identify and allow access to the item, +and allow users to access the appropriate bit encoding from there.

    +

    Of course, it may be that a +particular bit encoding of a file is explicitly being preserved; in +this case, the bitstream could be the only one in the item, and the +item's Handle would then essentially refer just to that bitstream. The +same bitstream can also be included in other items, and thus would be +citable as part of a greater item, or individually.

    +

    The Handle system also features +a global resolution infrastructure; that is, an end-user can enter a +Handle into any service (e.g. Web page) that can resolve Handles, and +the end-user will be directed to the object (in the case of DSpace, +community, collection or item) identified by that Handle. In order to +take advantage of this feature of the Handle system, a DSpace site must +also run a 'Handle server' that can accept and resolve incoming +resolution requests. All the code for this is included in the DSpace +source code bundle.

    +

    Handles can be written in two +forms:

    +
    hdl:1721.123/4567
    http://hdl.handle.net/1721.123/4567
    +

    The above represent the same +Handle. The first is possibly more convenient to use only as an +identifier; however, by using the second form, any Web browser becomes +capable of resolving Handles. An end-user need only access this form of +the Handle as they would any other URL. It is possible to enable some +browsers to resolve the first form of Handle as if they were standard +URLs using CNRI's +Handle Resolver plug-in, but +since the first form can always be simply derived from the second, +DSpace displays Handles in the second form, so that it is more useful +for end-users.

    +

    It is important to note that +DSpace uses the CNRI Handle infrastructure only at the 'site' level. +For example, in the above example, the DSpace site has been assigned +the prefix '1721.123'. It is still the responsibility of the DSpace +site to maintain the association between a full Handle (including the +'4567' local part) and the community, collection or item in question.

    +

    Bitstream 'Persistent' +Identifiers

    +

    As of DSpace 1.2, bitstreams in +DSpace also have more persistent identifiers. They are more volatile +than Handles, since if the content is moved to a different server or +organizaion, they will no longer work (hence the quotes around +'persistent'). However, they are more easily persisted than the simple +URLs based on database primary key previously used. This means that +external systems can more reliably refer to specific bitstreams stored +in a DSpace instance.

    +

    Each bitstream has a sequence +ID, unique within an item. This sequence ID is used to create a +persistent ID, of the form:

    +

    dspace +url/bitstream/handle/sequence +ID/filename

    +

    For example:

    +
    https://dspace.myu.edu/bitstream/123.456/789/24/foo.html
    +

    The above refers to the +bitstream with sequence ID 24 in the item with the Handle hdl:123.456/789. +The foo.html +is really just there as a hint to browsers: Although DSpace will +provide the appropriate MIME type, some browsers only function +correctly if the file has an expected extension.
    +

    +

    Storage Resource Broker (SRB) Support
    +

    +

    DSpace offers two means for +storing bitstreams. The first is in the file system on the server. The +second is using SRB (Storage +Resource +Broker). Both are achieved using +a simple, lightweight API.
    +

    +SRB is purely an option but may +be used in lieu of the server's file system or in addition to the file +system. Without going into a full description, SRB is a very robust, +sophisticated storage manager that offers essentially unlimited storage +and straightforward means to replicate (in simple terms, backup) the +content on other local or remote storage resources. +

    +

    Search and Browse

    +

    DSpace allows end-users to +discover content in a number of ways, including:

    + +

    Search is an essential +component of discovery in DSpace. Users' expectations from a search +engine are quite high, so a goal for DSpace is to supply as many search +features as possible. DSpace's indexing and search module has a very +simple API which allows for indexing new content, regenerating the +index, and performing searches on the entire corpus, a community, or +collection. Behind the API is the Java freeware search engine Lucene. +Lucene gives us fielded searching, stop word removal, stemming, and the +ability to incrementally add new indexed content without regenerating +the entire index.

    +

    Another important mechanism for +discovery in DSpace is the browse. This is the process whereby the user +views a particular index, such as the title index, and navigates around +it in search of interesting items. The browse subsystem provides a +simple API for achieving this by allowing a caller to specify an index, +and a subsection of that index. The browse subsystem then discloses the +portion of the index of interest. Indices that may be browsed are item +title, item issue date and authors. Additionally, the browse can be +limited to items within a particular collection or community.

    +

    HTML Support

    +

    For the most part, at present +DSpace simply supports uploading and downloading of bitstreams as-is. +This is fine for the majority of commonly-used file formats -- for +example PDFs, Microsoft Word documents, spreadsheets and so forth. HTML +documents (Web sites and Web pages) are far more complicated, and this +has important ramifications when it comes to digital preservation:

    + +

    Dealing with these issues is +the topic of much active research. Currently, DSpace bites off a small, +tractable chunk of this problem. DSpace can store and provide on-line +browsing capability for self-contained, +non-dynamic HTML documents. In +practical terms, this means:

    + +

    OAI Support

    +

    The Open Archives +Initiative has developed a protocol +for metadata harvesting. This +allows sites to programmatically retrieve or 'harvest' the metadata +from several sources, and offer services using that metadata, such as +indexing or linking services. Such a service could allow users to +access information from a large number of sites from one place.

    +

    DSpace exposes the Dublin Core +metadata for items that are publicly (anonymously) accessible. +Additionally, the collection structure is also exposed via the OAI +protocol's 'sets' mechanism. OCLC's open source OAICat +framework is used to provide this functionality.

    +

    DSpace's OAI service does +support the exposing of deletion information for withdrawn items, but +not for items that are 'expunged' (see above). +DSpace also supports OAI-PMH resumption tokens.

    +

    OpenURL Support

    +

    DSpace supports the OpenURL +protocol +from SFX, +in a rather simple fashion. If your institution has an SFX server, +DSpace will display an OpenURL link on every item page, automatically +using the Dublin Core metadata. Additionally, DSpace can respond to +incoming OpenURLs. Presently it simply passes the information in the +OpenURL to the search subsystem. A list of results is then displayed, +which usually gives the relevant item (if it is in DSpace) at the top +of the list.

    +

    Creative Commons Support

    +

    Dspace provides support for +Creative Commons licenses to be attached to items in the repository. +They represent an alternative to traditional copyright. To learn more +about Creative Commons, visit their +website. +Support for the licenses is controlled by a site-wide configuration +option, and since license selection involves redirection to the +Creative Commons website, additional parameters may be configured to +work with a proxy server. If the option is enabled, users may select a +Creative Commons license during the submission process, or elect to +skip Creative Commons licensing. If a selection is made a copy of the +license text and RDF metadata is stored along with the item in the +repository. There is also an indication - text and a Creative Commons +icon - in the item display page of the web user interface when an item +is licensed under Creative Commons.

    +

    Subscriptions

    +

    As noted above, +end-users (e-people) may 'subscribe' to collections in order to be +alerted when new items appear in those collections. Each day, end-users +who are subscribed to one or more collections will receive an e-mail +giving brief details of all new items that appeared in any of those +collections the previous day. If no new items appeared in any of the +subscribed collections, no e-mail is sent. Users can unsubscribe +themselves at any time.

    +

    History

    +

    While provenance information in +the form of prose is very useful, it is not easily programmatically +manipulated. The History system captures a time-based record of +significant changes in DSpace, in a manner suitable for later +'refactoring' or repurposing.

    +

    Currently, the History +subsystem is explicitly invoked when significant events occur (e.g., +DSpace accepts an item into the archive). The History subsystem then +creates RDF data describing the current state of the object. The RDF +data is modeled using Harmony/ABC, +an ontology for describing temporal-based data, and stored in the file +system. Some simple indices for unwinding the data are available.

    +

    Import +and Export

    +

    DSpace also includes batch +tools to import and export items in a simple directory structure, where +the Dublin Core metadata is stored in an XML file. This may be used as +the basis for moving content between DSpace and other systems.

    +

    There is also a METS-based +export tool, which exports items as METS-based metadata with associated +bitstreams referenced from the METS file.
    +

    +

    Registration

    +

    Registration is an alternate +means of incorporating items, their metadata, and their bitstreams into +DSpace by taking advantage of the bitstreams already being in +accessible computer storage. An example might be that there is a +repository for existing digital assets. Rather than using the normal +interactive +ingest process or the batch import +to furnish DSpace the metadata +and to upload bitstreams, registration provides DSpace the metadata and +the location +of the +bitstreams. DSpace uses a variation of the import tool to accomplish +registration.
    +

    +
    +
    Copyright © +2002-2004 MIT and Hewlett Packard
    + + diff --git a/dspace/docs/history.html b/dspace/docs/history.html new file mode 100644 index 0000000000..657c978fcd --- /dev/null +++ b/dspace/docs/history.html @@ -0,0 +1,535 @@ + + + + DSpace System Documentation: Version History + + + + +

    DSpace System Documentation: Version History

    +

    Back to contents

    + +

    Changes in DSpace 1.3beta1

    + +

    General Improvements

    + + + +

    Bug fixes

    + + +

    Changes in DSpace 1.2.2

    + +

    General Improvements

    + + + +

    Bug fixes

    + + +

    Changes in JSPs (since 1.2.1)

    + + + +

    Changes in DSpace 1.2.2

    + +

    General Improvements

    + + + +

    Bug fixes

    + + +

    Changes in JSPs

    + + + +

    Changes in DSpace 1.2.1

    + + +

    General Improvements

    + + + +

    Bug fixes

    + + +

    Changed JSPs

    + + + + +

    Changes in DSpace 1.2

    + +

    General Improvments

    + + + +

    Administration

    + + +

    Import/Export/OAI

    + + + +

    Miscellaneous

    + +
    +

    JSP file changes between 1.1 and 1.2

    +

    This list generated with cvs -Q rdiff -s -r dspace-1_1 dspace +and a sprinkling of perl.

    + +
    +
    +

    Java file changes between 1.1 and 1.2

    +

    This list generated with cvs -Q rdiff -s -r dspace-1_1 dspace +and a sprinkling of perl.

    + +
    +

    Changes in DSpace 1.1.1

    +

    Bug fixes

    + +

    Improvements

    + +

    Changes in DSpace 1.1

    + +
    +
    Copyright © 2002-2004 MIT and Hewlett Packard
    + + diff --git a/dspace/docs/image/architecture-600x450.gif b/dspace/docs/image/architecture-600x450.gif new file mode 100644 index 0000000000..fa1d5e2edf Binary files /dev/null and b/dspace/docs/image/architecture-600x450.gif differ diff --git a/dspace/docs/image/data-model.gif b/dspace/docs/image/data-model.gif new file mode 100644 index 0000000000..539eedf1a2 Binary files /dev/null and b/dspace/docs/image/data-model.gif differ diff --git a/dspace/docs/image/ingest.gif b/dspace/docs/image/ingest.gif new file mode 100644 index 0000000000..988f76d65f Binary files /dev/null and b/dspace/docs/image/ingest.gif differ diff --git a/dspace/docs/image/web-ui-flow.gif b/dspace/docs/image/web-ui-flow.gif new file mode 100644 index 0000000000..99b4c8f453 Binary files /dev/null and b/dspace/docs/image/web-ui-flow.gif differ diff --git a/dspace/docs/image/workflow.gif b/dspace/docs/image/workflow.gif new file mode 100644 index 0000000000..2ba45fff15 Binary files /dev/null and b/dspace/docs/image/workflow.gif differ diff --git a/dspace/docs/index.html b/dspace/docs/index.html new file mode 100644 index 0000000000..e3af4f3047 --- /dev/null +++ b/dspace/docs/index.html @@ -0,0 +1,135 @@ + + + + DSpace System Documentation: Contents + + + + +

    DSpace System Documentation: Contents

    + +

    Authors: Robert Tansley, Mick Bass, Margret Branschofsky, Grace Carpenter, Greg McClellan, David Stuve

    + +

    DSpace Version: 1.3beta1 (28-Jun-2005)

    +

    Documentation Version: 1.3beta1 (28-Jun-2005)

    +

    Documentation for other versions of DSpace may be downloaded from SourceForge

    + + +
    + +
    + Copyright © 2002-2005 MIT and Hewlett Packard +
    + + diff --git a/dspace/docs/install.html b/dspace/docs/install.html new file mode 100644 index 0000000000..0a17ca7dfb --- /dev/null +++ b/dspace/docs/install.html @@ -0,0 +1,336 @@ + + + + DSpace System Documentation: Installation + + + + +

    DSpace System Documentation: Installation

    + +

    Back to contents

    + + +

    Prerequisites

    + +

    The list below describes the third-party components and tools you'll need to run a DSpace server. These are simply recommendations based on our setup at MIT; since DSpace is built on open source, standards-based tools, there are numerous other possibilities and setups.

    + +

    Also, please note that the configuration and installation guidelines relating to a particular tool below are here for convenience. You should refer to the documentation for each individual component for complete and up-to-date details. Many of the tools are updated on a frequent basis, and the guidelines below may become out of date.

    + +
      +

    1. UNIX-like OS (Linux, HP/UX etc)

      2. + +

    2. Java 1.4 or later (standard SDK is fine, you don't need J2EE)

      3. + +

    3. Apache Ant 1.5 or later (Java make-like tool)

      4. + +
    4. +

      PostgreSQL 7.3 or later, an open source relational database.

      + +

      Be sure to compile with the following options to the 'configure' script:

      + + 
      --enable-multibyte --enable-unicode --with-java
      + +

      Once installed, you need to enable TCP/IP connections (DSpace uses JDBC). Edit postgresql.conf (usually in /usr/local/pgsql/data or /var/lib/pgsql/data), and add this line:

      + + 
      tcpip_socket = true
      + +

      Then tighten up security a bit by editing pg_hba.conf and adding this line:

      + + 
      host  dspace  dspace  127.0.0.1  255.255.255.255  md5
      + +

      Then restart PostgreSQL.

      +
      5. + +

    5. Jakarta Tomcat 4.x/5.x or equivalent, such as Jetty or Caucho Resin.

      + +

      Note that DSpace will need to run as the same user as Tomcat, so you might want to install and run Tomcat as a user called 'dspace'.

      + +

      You need to ensure that Tomcat has a) enough memory to run DSpace and b) uses UTF-8 as its default file encoding for international character support. So ensure in your startup scripts (etc) that the following environment variable is set:

      + + 
      JAVA_OPTS="-Xmx512M -Xms64M -Dfile.encoding=UTF-8"
      + +

      You also need to alter Tomcat's default configuration to support searching and browsing of multi-byte UTF-8 correctly. You need to add a configuration option to the <Connector> element in [tomcat]/config/server.xml:

      + + 
      URIEncoding="UTF-8"
      + +

      e.g. if you're using the default Tomcat config, it should read:

      + + 
      <!-- Define a non-SSL HTTP/1.1 Connector on port 8080 -->
+<Connector port="8080"
+           maxThreads="150" minSpareThreads="25" maxSpareThreads="75"
+           enableLookups="false" redirectPort="8443" acceptCount="100"
+           connectionTimeout="20000" disableUploadTimeout="true"
+           URIEncoding="UTF-8" />
      + +

      Jetty and Resin are configured for correct handling of UTF-8 by default.

      +
      6. +
    + + +

    Quick Installation Steps

    + +

    But First, a Word on Directories and Path Names

    + +

    DSpace uses three separate directory trees. Although you don't need to know all the details +of them in order to install DSpace, you do need to know they exist and also know how they're referred to in this document:

    +

    + +

    For details on the contents of these separate directory trees, refer to + directories.html. + Note that the source directory and install directory should always be separate!

    + +
      +
    1. +

      Create the DSpace user. This needs to be the same user that Tomcat (or Jetty etc) will run as. e.g. as root run:

      + + 
      useradd -m dspace
      +
      2. + +
    2. +

      Download the latest DSpace source code release and unpack it:

      + + 
      gunzip -c dspace-source-1.x.tar.gz | tar -xf -
      + +
      3. + +
    3. +

      Copy the PostgreSQL JDBC driver (.jar file) into +[dspace-source]/lib. If you compiled PostgreSQL yourself, it'll be in postgresql-7.x.x/src/interfaces/jdbc/jars/postgresql.jar. Alternatively you can download it directly from the PostgreSQL JDBC site. Make sure you get the driver for the version of PostgreSQL you're running and for JDBC2.

      +
      4. + +
    4. +

      Create a dspace database, owned by the dspace PostgreSQL user:

      + + 
      createuser -U postgres -d -A -P dspace ; createdb -U dspace -E UNICODE dspace
      + +

      Enter a password for the DSpace database. (This isn't the same as the dspace user's UNIX password.)

      +
      5. + +
    5. +

      Edit [dspace-source]/config/dspace.cfg, in particular you'll need to set these properties:

      + + 
      dspace.url
+dspace.hostname
+dspace.name
+db.password   (the password you entered in the previous step)
+mail.server
+mail.from.address
+feedback.recipient
+mail.admin
+alert.recipient (not essential but very useful!)
      + +

      Note that if you change dspace.dir you'll also need to change other properties with values that start with /dspace, e.g. assetstore.dir, log.dir...

      +
      6. + +
    6. +

      Create the directory for the DSpace installation. As root, run:

      + + 
      mkdir [dspace] ; chown dspace [dspace]
      + +

      (Assuming the dspace UNIX username.)

      +
      7. + +
    7. +

      As the dspace UNIX user, compile and install DSpace:

      + + 
      cd [dspace-source] ; ant fresh_install
      + +

      The most likely thing to go wrong here is the database connection. See the common problems section.

      +
      8. + +
    8. +

      Copy the DSpace Web application archives (.war files) to the appropriate directory in your Tomcat/Jetty/Resin installation. For example:

      + + 
      cp [dspace-source]/build/*.war [tomcat]/webapps
      +
      9. + +
    9. +

      Create an initial administrator account:

      + + 
      [dspace]/bin/create-administrator
      +
      10. + +
    10. +

      Now the moment of truth! Start up (or restart) Tomcat. Visit the base URL of your server, e.g. http://dspace.myu.edu:8080/dspace. You should see the DSpace home page. Congratulations!

      +
      11. +
    + +

    In order to set up some communities and collections, you'll need to access the administration UI. To do this, append 'admin' to your server's URL, e.g. http://dspace.myu.edu:8080/dspace/dspace-admin.

    + +

    Advanced Installation

    + +

    The above installation steps are sufficient to set up a test server to play around with, but there are a few other steps and options you should probably consider before deploying a DSpace production site.

    + +

    'cron' Jobs

    + +

    A couple of DSpace features require that a script is run regularly -- the e-mail subscription feature that alerts users of new items being deposited, and the new 'media filter' tool, that generates thumbnails of images and extracts the full-text of documents for indexing.

    + +

    To set these up, you just need to run the following command as the dspace UNIX user:

    + + 
    crontab -e
    + +

    Then add the following lines:

    + + 
    # Send out subscription e-mails at 01:00 every day
+0 1 * * * [dspace]/bin/sub-daily
+# Run the media filter at 02:00 every day
+0 2 * * * [dspace]/bin/filter-media
    + +

    Naturally you should change the frequencies to suit your environment.

    + +

    PostgreSQL also benefits from regular 'vacuuming', which optimizes the indices and clears out any deleted data. Become the postgres UNIX user, run crontab -e and add (for example): + + 

    # Clean up the database nightly at 2.40am
+40 2 * * * vacuumdb --analyze dspace > /dev/null 2>&1
    + +

    DSpace over HTTPS

    + +

    Plain old HTTP is totally insecure, and if your DSpace uses username/password authentication or stores some restricted content, running it over HTTPS (HTTP over a Secure Socket Layer (SSL)) is advisable. There are two options for this: Using Apache HTTPD, or Tomcat/Jetty's in-built HTTPS support.

    + +

    To use Apache HTTPD: The DSpace source bundle includes a partial Apache configuration apache13.conf, which contains most of the DSpace-specific configuration required. It assumes you're using mod_webapp, which is deprecated and tricky to compile but a lot easier to configure than mod_jk2 which is the current recommendation from Tomcat. Use of this is optional, you might just want to use it as an example. To use it directly, in the main Apache httpd.conf, you should:

    + + + +

    To use Tomcat or Jetty's HTTPS support consult the documentation for the relevant tool. Also, these alternative DSpace install docs briefly describe getting Tomcat running with SSL.

    + + +

    The Handle Server

    + +

    First a few facts to clear up some common misconceptions:

    + + + +

    If you want to use the Handle system, you'll need to set up a Handle server. This is included with DSpace. Note that this is not required in order to evaluate DSpace; you only need one if you are running a production service. You'll need to obtain a Handle prefix from the central CNRI Handle site.

    + +

    A Handle server runs as a separate process that receives TCP requests from other Handle servers, and issues resolution requests to a global server or servers if a Handle entered locally does not correspond to some local content. The Handle protocol is based on TCP, so it will need to be installed on a server that can broadcast and receive TCP on port 2641.

    + +

    The Handle server code is included with the DSpace code in +[dspace-source]/lib/handle.jar. A script exists to create a simple Handle configuration - simply run [dspace]/bin/make-handle-config after you've set the appropriate parameters in dspace.cfg. You can also create a Handle configuration directly by following the installation instructions on handle.net, but with these changes:

    + + + +

    Whichever approach you take, start the Handle server with [dspace]/bin/start-handle-server, as the DSpace user. You will need to send the sitebndl.zip file to hdladmin@cnri.reston.va.us as described in the Handle server documentation.

    + +

    Note that since the DSpace code manages individual Handles, administrative operations such as Handle creation and modification aren't supported by DSpace's Handle server.

    + + +

    Checking Your Installation

    +

    TODO

    + +

    Known Bugs

    + +

    In any software project of the scale of DSpace, there will be bugs. Sometimes, a stable version of DSpace includes known bugs. We do not always wait until every known bug is fixed before a release. If the software is sufficiently stable and an improvement on the previous release, and the bugs are minor and have known workarounds, we release it to enable the community to take advantage of those improvements.

    + +

    The known bugs in a release are documented in the KNOWN_BUGS file in the source package.

    + +

    Please see the DSpace bug tracker for further information on current bugs, and to find out if the bug has subsequently been fixed. This is also where you can report any further bugs you find.

    + + +

    Common Problems

    + +

    In an ideal world everyone would follow the above steps and have a fully functioning DSpace. Of couse, in the real world it doesn't always seem to work out that way. This section lists common problems that people encounter when installing DSpace, and likely causes and fixes. This is likely to grow over time as we learn about users' experiences.

    + +
    +
    Database errors occur when you run ant fresh_install
    + +
    +

    There are two common errors that occur. If your error looks like this--

    + + 
    [java] 2004-03-25 15:17:07,730 INFO  org.dspace.storage.rdbms.InitializeDatabase @ Initializing Database
+[java] 2004-03-25 15:17:08,816 FATAL org.dspace.storage.rdbms.InitializeDatabase @ Caught exception:
+[java] org.postgresql.util.PSQLException: Connection refused. Check that the hostname and port are correct and that the postmaster is accepting TCP/IP connections.
+[java]     at org.postgresql.jdbc1.AbstractJdbc1Connection.openConnection(AbstractJdbc1Connection.java:204)
+[java]     at org.postgresql.Driver.connect(Driver.java:139)
    + +

    it usually means you haven't yet added the relevant configuration parameter to your PostgreSQL configuration (see above), or perhaps you haven't restarted PostgreSQL after making the change. +Also, make sure that the db.username and db.password properties are correctly set in +[dspace-source]/config/dspace.cfg.

    + +

    An easy way to check that your DB is working OK over TCP/IP is to try this on the command line:

    + + 
    psql -U dspace -W -h localhost
    + +

    Enter the dspace database password, and you should be dropped into the psql tool with a dspace=> prompt.

    + +

    Another common error looks like this:

    + + 
    [java] 2004-03-25 16:37:16,757 INFO  org.dspace.storage.rdbms.InitializeDatabase @ Initializing Database
+[java] 2004-03-25 16:37:17,139 WARN  org.dspace.storage.rdbms.DatabaseManager @ Exception initializing DB pool
+[java] java.lang.ClassNotFoundException: org.postgresql.Driver
+[java]     at java.net.URLClassLoader$1.run(URLClassLoader.java:198)
+[java]     at java.security.AccessController.doPrivileged(Native Method)
+[java]     at java.net.URLClassLoader.findClass(URLClassLoader.java:186)
    + +

    This means that the PostgreSQL JDBC driver is not present in [dspace-source]/lib. See above.

    + + +
    Tomcat doesn't shut down
    +

    If you're trying to tweak Tomcat's configuration but nothing seems to make a difference to the error you're seeing, you might find that Tomcat hasn't been shutting down properly, perhaps because it's waiting for a stale connection to close gracefully which won't happen. To see if this is the case, try:

    + + 
    ps -ef | grep java
    + +

    and look for Tomcat's Java processes. If they stay arround after running Tomcat's shutdown.sh script, trying killing them (with -9 if necessary), then starting Tomcat again.

    + +
    Database connections don't work, or accessing DSpace takes forever
    +

    If you find that when you try to access a DSpace Web page and your browser sits there connecting, or if the database connections fail, you might find that a 'zombie' database connection is hanging around preventing normal operation. To see if this is the case, try:

    + + 
    ps -ef | grep postgres
    + +

    You might see some processes like this

    + + 
    dspace 16325  1997  0  Feb 14  ?         0:00 postgres: dspace dspace 127.0.0.1 idle in transaction
    + +

    This is normal--DSpace maintains a 'pool' of open database connections, which are re-used to avoid the overhead of constantly opening and closing connections. If they're 'idle' it's OK; they're waiting to be used. However sometimes, if something went wrong, they might be stuck in the middle of a query, which seems to prevent other connections from operating, e.g.:

    + + 
    dspace 16325  1997  0  Feb 14  ?         0:00 postgres: dspace dspace 127.0.0.1 SELECT
    + +

    This means the connection is in the middle of a SELECT operation, and if you're not using DSpace right that instant, it's probably a 'zombie' connection. If this is the case, try killing the process, and stopping and restarting Tomcat.

    + +
    You've made changes to the code or to the JSP's and rebuilt DSpace successfully, but when you run Tomcat + you don't see any of your changes in DSpace.
    + +

    After you've rebuilt DSpace and copied dspace.war from your [dspace-source]/build directory + into your [tomcat]/webapps directory, you must + also delete the existing [tomcat]/webapps/dspace directory before re-starting Tomcat. Otherwise + Tomcat will continue to use the old code.

    + +
    + +
    + +
    + Copyright © 2002-2004 MIT and Hewlett Packard +
    + + diff --git a/dspace/docs/introduction.html b/dspace/docs/introduction.html new file mode 100644 index 0000000000..41ad1ae641 --- /dev/null +++ b/dspace/docs/introduction.html @@ -0,0 +1,48 @@ + + + + DSpace System Documentation: Introduction + + + + +

    DSpace System Documentation: Introduction

    + +

    Back to contents

    + +

    DSpace is an open source software platform that enables institutions to:

    + + + +

    This system documentation includes a functional overview of the system, which is a good introduction to the capabilities of the system, and should be readable by non-technical folk. Everyone should read this section first because it introduces some terminology used throughout the rest of the documentation.

    + +

    For people actually running a DSpace service, there is + an installation guide, and sections on configuration and the directory structure. Note that as of DSpace 1.2, the administration user interface guide is now on-line help available from within the DSpace system.

    + +

    Finally, for those interested in the details of how DSpace works, and those potentially interested in modifying the code for their own purposes, there is a detailed architecture and design section.

    + +

    Other good sources of information are:

    + + + +
    + +
    + Copyright © 2002-2005 MIT and Hewlett Packard +
    + + diff --git a/dspace/docs/make-doc-package b/dspace/docs/make-doc-package new file mode 100755 index 0000000000..525ea9dbed --- /dev/null +++ b/dspace/docs/make-doc-package @@ -0,0 +1,36 @@ +#!/bin/sh + +USAGE="$0 cvs-tag version" + +# Just in case you need to 'socksify' etc +CVS_COMMAND="cvs" + +# Check args +if [ "$#" != "2" ]; then + echo $USAGE + exit 1 +fi + +FILENAME="dspace-$2-docs" + + +mkdir tmp +cd tmp + +echo "Checking out docs..." +$CVS_COMMAND -q export -r $1 docs + +# Remove stuff not to include in ZIP +rm -f docs/.cvsignore +rm -rf docs/originals +rm -f docs/make-doc-package + +echo "Creating ZIP..." +mv docs $FILENAME +zip -9 -r ../$FILENAME.zip $FILENAME + +echo "Cleaning up..." +cd .. +rm -rf tmp + +echo "Package created as $FILENAME.zip" diff --git a/dspace/docs/originals/architecture.vsd b/dspace/docs/originals/architecture.vsd new file mode 100644 index 0000000000..612e85f897 Binary files /dev/null and b/dspace/docs/originals/architecture.vsd differ diff --git a/dspace/docs/originals/data-model.vsd b/dspace/docs/originals/data-model.vsd new file mode 100644 index 0000000000..b25dce0dc2 Binary files /dev/null and b/dspace/docs/originals/data-model.vsd differ diff --git a/dspace/docs/originals/ingest.vsd b/dspace/docs/originals/ingest.vsd new file mode 100644 index 0000000000..76d38d005c Binary files /dev/null and b/dspace/docs/originals/ingest.vsd differ diff --git a/dspace/docs/originals/web-ui-flow.vsd b/dspace/docs/originals/web-ui-flow.vsd new file mode 100644 index 0000000000..9c4b1748ed Binary files /dev/null and b/dspace/docs/originals/web-ui-flow.vsd differ diff --git a/dspace/docs/originals/workflow.vsd b/dspace/docs/originals/workflow.vsd new file mode 100644 index 0000000000..791a3791b2 Binary files /dev/null and b/dspace/docs/originals/workflow.vsd differ diff --git a/dspace/docs/postgres-upgrade-notes.txt b/dspace/docs/postgres-upgrade-notes.txt new file mode 100644 index 0000000000..55976321f2 --- /dev/null +++ b/dspace/docs/postgres-upgrade-notes.txt @@ -0,0 +1,117 @@ +Updating Postgres with a DSpace installation. + + + + +1. Build new postgres. + Be sure to run configure with at least these options: + ./configure --enable-multibyte --enable-unicode --with-java + +2. shutdown tomcat + +3. dump current data + pg_dumpall -o >dspace.out + +4. shut down postgres + pg_ctl stop -D /dspace/database/data -m fast + +5. back up old data directory + mv /dspace/database/data /dspace/database/data.old + +6. install new postgres + +7. start new postgres + initdb -D /dspace/database/data + + edit /dspace/database/data/postgresql.conf (Add 'tcpip_socket = true') + + pg_ctl start -D /dspace/database/data + +8. restore data + psql -d template1 -f dspace.out + +9. Install new JDBC driver + from the new postgres installation directory: + cp share/java/postgres.jar /dspace/lib + +10. restart tomcat + + +------------------------------------------------------------------------------- +Notes from postgres install docs: +------------------------------------------------------------------------------- + + If You Are Upgrading + +The internal data storage format changes with new releases of PostgreSQL. +Therefore, if you are upgrading an existing installation that does not have a +version number "7.3.x", you must back up and restore your data as shown here. +These instructions assume that your existing installation is under the "/usr/ +local/pgsql" directory, and that the data area is in "/usr/local/pgsql/data". +Substitute your paths appropriately. + + 1. Make sure that your database is not updated during or after the backup. + This does not affect the integrity of the backup, but the changed data + would of course not be included. If necessary, edit the permissions in + the file "/usr/local/pgsql/data/pg_hba.conf" (or equivalent) to disallow + access from everyone except you. + + 2. To back up your database installation, type: + + pg_dumpall > outputfile + + If you need to preserve OIDs (such as when using them as foreign keys), + then use the "-o" option when running "pg_dumpall". + "pg_dumpall" does not save large objects. Check the Administrator's Guide + if you need to do this. + To make the backup, you can use the "pg_dumpall" command from the version + you are currently running. For best results, however, try to use the + "pg_dumpall" command from PostgreSQL 7.3.1, since this version contains + bug fixes and improvements over older versions. While this advice might + seem idiosyncratic since you haven't installed the new version yet, it is + advisable to follow it if you plan to install the new version in parallel + with the old version. In that case you can complete the installation + normally and transfer the data later. This will also decrease the + downtime. + + 3. If you are installing the new version at the same location as the old one + then shut down the old server, at the latest before you install the new + files: + + kill -INT `cat /usr/local/pgsql/data/postmaster.pid` + + Versions prior to 7.0 do not have this "postmaster.pid" file. If you are + using such a version you must find out the process id of the server + yourself, for example by typing "ps ax | grep postmaster", and supply it + to the "kill" command. + On systems that have PostgreSQL started at boot time, there is probably a + start-up file that will accomplish the same thing. For example, on a Red + Hat Linux system one might find that + + /etc/rc.d/init.d/postgresql stop + + works. Another possibility is "pg_ctl stop". + + 4. If you are installing in the same place as the old version then it is + also a good idea to move the old installation out of the way, in case you + have trouble and need to revert to it. Use a command like this: + + mv /usr/local/pgsql /usr/local/pgsql.old + +After you have installed PostgreSQL 7.3.1, create a new database directory and +start the new server. Remember that you must execute these commands while +logged in to the special database user account (which you already have if you +are upgrading). + + /usr/local/pgsql/bin/initdb -D /usr/local/pgsql/data + /usr/local/pgsql/bin/postmaster -D /usr/local/pgsql/data + +Finally, restore your data with + + /usr/local/pgsql/bin/psql -d template1 -f outputfile + +using the *new* psql. +These topics are discussed at length in the Administrator's Guide, which you +are encouraged to read in any case. + +------------------------------------------------------------------------------- \ No newline at end of file diff --git a/dspace/docs/storage.html b/dspace/docs/storage.html new file mode 100644 index 0000000000..1d017f5e81 --- /dev/null +++ b/dspace/docs/storage.html @@ -0,0 +1,383 @@ + + + + DSpace System Documentation: Storage Layer + + + + +

    DSpace +System Documentation: +Storage Layer

    +

    Back to contents
    +Back +to architecture overview

    +

    RDBMS

    +

    DSpace uses a relational +database to store all information about the organization of content, +metadata about the content, information about e-people and +authorization, and the state of currently-running workflows. The DSpace +system also uses the relational database in order to maintain indices +that users can browse.

    +

    Most of the functionality that +DSpace uses can be offered by any standard SQL database that supports +transactions. Presently, the browse indices use some features specific +to PostgreSQL, +a mature open-source relational database, so some modification to the +code would be needed before DSpace would function fully with an +alternative database back-end.

    +

    The org.dspace.storage.rdbms +package provides access to an SQL database in a somewhat simpler form +than using JDBC directly. The main class is DatabaseManager, +which executes SQL queries and returns TableRow +or TableRowIterator +objects. The InitializeDatabase +class is used to load SQL into the database via JDBC, for example to +set up the schema.

    +

    All calls to the Database +Manager require a DSpace Context +object. Example use of the +database manager API is given in the org.dspace.storage.rdbms +package Javadoc.

    +

    The database schema used by +DSpace is stored in [dspace-source]/etc/database_schema.sql +in the source distribution. It is stored in the form of SQL that can be +fed straight into the DBMS to construct the database. The schema SQL +file also directly creates two e-person groups in the database that are +required for the system to function properly.

    +

    The DSpace database code uses +an SQL function getnextid +to assign primary keys to newly created rows. This SQL function must be +safe to use if several JVMs are accessing the database at once; for +example, the Web UI might be creating new rows in the database at the +same time as the batch item importer. The PostgreSQL-specific +implementation of the method uses SEQUENCES +for each table in order to create new IDs. If an alternative database +backend were to be used, the implementation of getnextid +could be updated to operate with that specific DBMS.

    +

    The etc +directory in the source distribution contains two further SQL files. clean-database.sql +contains the SQL necessary to completely clean out the database, so use +with caution! The Ant target clean_database +can be used to execute this. update-sequences.sql +contains SQL to reset the primary key generation sequences to +appropriate values. You'd need to do this if, for example, you're +restoring a backup database dump which creates rows with specific +primary keys already defined. In such a case, the sequences would +allocate primary keys that were already used.

    +

    Maintenance and Backup

    +

    When using PostgreSQL, it's a +good idea to perform regular 'vacuuming' of the database to optimize +performance. This is performed by the vacuumdb +command which can be executed via a 'cron' job, for example by putting +this in the system crontab:

    +
    # clean up the database nightly
    40 2 * * * /usr/local/pgsql/bin/vacuumdb --analyze dspace > /dev/null 2>&1
    +

    The DSpace database can be +backed up and restored using usual methods, for example with pg_dump +and psql. +However when restoring a database, you will need to perform these +additional steps:

    + +

    Future updates of DSpace may +involve minor changes to the database schema. Specific instructions on +how to update the schema whilst keeping live data will be included. The +current schema also contains a few currently unused database columns, +to be used for extra functionality in future releases. These unused +columns have been added in advance to minimize the effort required to +upgrade.

    +

    Configuring the RDBMS Component

    +

    The database manager is +configured with the following properties in dspace.cfg:

    + + + + + + + + + + + + + + + + + + + +
    db.urlThe JDBC URL to use for +accessing the database. This should not point to a connection pool, +since DSpace already implements a connection pool.
    db.driverJDBC driver class name. +Since presently, DSpace uses PostgreSQL-specific features, this should +be org.postgresql.Driver.
    db.usernameUsername to use when +accessing the database.
    db.passwordCorresponding password +ot use when accessing the database.
    +

    Bitstream Store

    +

    DSpace offers two means for +storing content. The first is in the file system on the server. The +second is using SRB (Storage +Resource +Broker). Both are achieved using +a simple, lightweight API.
    +

    +

    SRB is purely an option but may +be used in lieu of the server's file system or in addition to the file +system. Without going into a full description, SRB is a very robust, +sophisticated storage manager that offers essentially unlimited storage +and straightforward means to replicate (in simple terms, backup) the +content on other local or remote storage resources.
    +

    +

    The terms "store", "retrieve", +"in the system", "storage", and so forth, used below can refer to +storage in the file system on the server ("traditional") or in SRB.
    +

    +

    The BitstreamStorageManager +provides low-level access to bitstreams stored in the system. In +general, it should not be used directly; instead, use the Bitstream +object in the content management API +since that encapsulated authorization and other metadata to do with a +bitstream that are not maintained by the BitstreamStorageManager.

    +

    The bitstream storage manager +provides three methods that store, retrieve and delete bitstreams. +Bitstreams are referred to by their 'ID'; that is the primary key bitstream_id +column of the corresponding row in the database.

    +

    As of DSpace version 1.1, there +can be multiple bitstream stores. Each of these bitstream stores can be +traditional storage or SRB storage. This means that the potential +storage of a DSpace system is not bound by the maximum size of a single +disk or file system and also that traditional and SRB storage can be +combined in one DSpace installation. Both traditional and SRB storage +are specified by configuration +parameters. Also see Configuring +the Bitstream Store below.
    +

    +

    Stores are numbered, starting +with zero, then counting upwards. Each bitstream entry in the database +has a store number, used to retrieve the bitstream when required.

    +

    At the moment, the store in +which new bitstreams are placed is decided using a configuration +parameter, and there is no provision for moving bitstreams between +stores. Administrative tools for manipulating bitstreams and stores +will be provided in future releases. Right now you can move a whole +store (e.g. you could move store number 1 from /localdisk/store +to /fs/anotherdisk/store +but it would still have to be store number 1 and have the exact same +contents.

    +

    Bitstreams also have an +38-digit internal ID, different from the primary key ID of the +bitstream table row. This is not visible or used outside of the +bitstream storage manager. It is used to determine the exact location +(relative to the relevant store directory) that the bitstream is stored +in traditional or SRB storage. The first three pairs of digits are the +directory path that the bitstream is stored under. The bitstream is +stored in a file with the internal ID as the filename.

    +

    For example, a bitstream with +the internal ID 12345678901234567890123456789012345678 +is stored in the directory:

    +
    (assetstore dir)/12/34/56/12345678901234567890123456789012345678
    +

    The reasons for storing files +this way are:

    + +

    When storing a bitstream, the BitstreamStorageManager +DOES set the following fields in the corresponding database table row:

    + +

    The remaining fields are the +responsibility of the Bitstream +content management API class.

    +

    The bitstream storage manager +is fully transaction-safe. In order to implement transaction-safety, +the following algorithm is used to store bitstreams:

    +
      +
    1. A database connection is +created, separately from the currently active connection in the current DSpace context.
      2. +
    2. An unique internal +identifier (separate from the database primary key) is generated.
      3. +
    3. The bitstream DB table row +is created using this new connection, with the deleted +column set to true.
      4. +
    4. The new connection is committed, +so the 'deleted' bitstream row is written to the database
      5. +
    5. The bitstream itself is +stored in a file in the configured 'asset store directory', with a +directory path and filename derived from the internal ID
      6. +
    6. The deleted +flag in the bitstream row is set to false. +This will occur (or not) as part of the current DSpace Context.
      7. +
    +

    This means that should anything +go wrong before, during or after the bitstream storage, only one of the +following can be true:

    + +

    None of these affect the +integrity of the data in the database or bitstream store.

    +

    Similarly, when a bitstream is +deleted for some reason, its deleted +flag is set to true as part of the overall transaction, and the +corresponding file in storage is not +deleted.

    +

    The above techniques mean that +the bitstream storage manager is transaction-safe. Over time, the +bitstream database table and file store may contain a number of +'deleted' bitstreams. The cleanup +method of BitstreamStorageManager +goes through these deleted rows, and actually deletes them along with +any corresponding files left in the storage. It only removes 'deleted' +bitstreams that are more than one hour old, just in case cleanup is +happening in the middle of a storage operation.

    +

    This cleanup can be invoked +from the command line via the Cleanup +class, which can in turn be easily executed from a shell on the server +machine using /dspace/bin/cleanup. +You might like to have this run regularly by cron, +though since DSpace is read-lots, write-not-so-much it doesn't need to +be run very often.

    +

    Backup

    +

    The bitstreams (files) in +traditional storage may be backed up very easily by simply 'tarring' or +'zipping' the assetstore +directory (or whichever directory is configured in dspace.cfg). +Restoring is as simple as extracting the backed-up compressed file in +the appropriate location.
    +

    +

    Similar means could be used for +SRB, but SRB offers many more options for managing backup.
    +

    +

    It is important to note that +since the bitstream storage manager holds the bitstreams in storage, +and information about them in the database, that a database backup and +a backup of the files in the bitstream store must be made at the same +time; the bitstream data in the database must correspond to the stored +files.

    +

    Of course, it isn't really +ideal to 'freeze' the system while backing up to ensure that the +database and files match up. Since DSpace uses the bitstream data in +the database as the authoritative record, it's best to back up the +database before the files. This is because it's better to have a +bitstream in storage but not the database (effectively non-existent to +DSpace) than a bitstream record in the database but not storage, since +people would be able to find the bitstream but not actually get the +contents.

    +

    Configuring the Bitstream Store

    +Both traditional and SRB bitstream stores are configured in dspace.cfg. +

    Configuring Traditonal Storage

    +Bitstream stores in the file system on the server are configured like +this:
    +     +
    assetstore.dir = [dspace]/assetstore
    +

    (Remember that [dspace] +is a placeholder for the actual name of your DSpace install directory).

    +

    The above example specifies a +single asset store.

    +
    assetstore.dir = [dspace]/assetstore_0
    assetstore.dir.1 = /mnt/other_filesystem/assetstore_1
    +

    The above example specifies two +asset stores. assetstore.dir specifies the asset store number 0 (zero); +after that use assetstore.dir.1, assetstore.dir.2 and so on. The +particular asset store a bitstream is stored in is held in the +database, so don't move bitstreams between asset stores, and don't +renumber them.

    +

    By default, newly created +bitstreams are put in asset store 0 (i.e. the one specified by the +assetstore.dir property.) This allows backwards compatibility with +pre-DSpace 1.1 configurations. To change this, for example when asset +store 0 is getting full, add a line to dspace.cfg +like:

    +
    assetstore.incoming = 1
    +

    Then restart DSpace (Tomcat). +New bitstreams will be written to the asset store specified by assetstore.dir.1, +which is /mnt/other_filesystem/assetstore_1 +in the above example.
    +

    +

    Configuring SRB Storage

    +The same framework is used to configure SRB storage. That is, the asset +store number (0..n) can reference a file system directory as above or +it can reference a set +of SRB account parameters. But any particular asset store number can +reference one or the other but not both. This way traditional and SRB +storage can both be used but with different asset store numbers. The +same cautions mentioned above apply to SRB asset stores as well: The +particular asset store a bitstream is stored in is held in the +database, so don't move bitstreams between asset stores, and don't +renumber them.
    +
    +For example, let's say asset store number 1 will refer to SRB. The +there will be a set of SRB account parameters like this:
    +
    srb.host.1 = mysrbmcathost.myu.edu
    srb.port.1 = 5544
    srb.mcatzone.1 = mysrbzone
    srb.mdasdomainname.1 = mysrbdomain
    srb.defaultstorageresource.1 = mydefaultsrbresource
    srb.username.1 = mysrbuser
    srb.password.1 = mysrbpassword
    srb.homedirectory.1 = /mysrbzone/home/mysrbuser.mysrbdomain
    srb.parentdir.1 = mysrbdspaceassetstore
    +Several of the terms, such as mcatzone, +have meaning only in the SRB context and will be familiar to SRB users. +The last, srb.parentdir.n, +can be used to used for addition (SRB) upper directory structure within +an SRB account. This property value could be blank as well.
    +
    +(If asset store 0 would refer to SRB it would be srb.host = ..., +srb.port = ..., and so on (.0 omitted) to be consistent +with the traditional storage configuration above.)
    +
    +The similar use of assetstore.incoming +to reference asset store 0 (default) or 1..n (explicit property) means +that new bitstreams will be written to traditional or SRB storage +determined by whether a file system directory on the server is +referenced or a set of SRB account parameters are referenced.
    +
    +There are comments in dspace.cfg that further elaborate the +configuration of traditional and SRB storage.
    +
    +
    +
    Copyright © +2002-2004 MIT and Hewlett Packard
    + + diff --git a/dspace/docs/style.css b/dspace/docs/style.css new file mode 100644 index 0000000000..2330553e9b --- /dev/null +++ b/dspace/docs/style.css @@ -0,0 +1,29 @@ +BODY { font-family: "verdana", Arial, Helvetica, sans-serif; + font-size: 10pt; + font-style: normal; + color: #000000; + background: #ffffff; + margin: 30px } + +P { text-align: justify } + +H1 { text-align: center } + +TABLE { text-align: center } + +TH { text-align: center; + font-size: 10pt; + font-weight: bold } + +TD { text-align: left; + font-size: 10pt; + padding: 4px } + +DT { font-weight: bold } + +.figure { text-align: center; + margin-bottom: 2px } + +.caption { text-align: center; + margin-top: 0; + font-size: 8pt } \ No newline at end of file diff --git a/dspace/docs/submission.html b/dspace/docs/submission.html new file mode 100644 index 0000000000..46a193c10d --- /dev/null +++ b/dspace/docs/submission.html @@ -0,0 +1,274 @@ + + + + + + + DSpace System Documentation: Submission Forms Customization + + + + + +

    Custom Metadata-entry Pages for Submission

    + +

    Back to contents

    + +

    Introduction

    + +

    This section explains how to customize the Web forms used by submitters and editors to enter and modify the metadata for a new item.

    + +

    You can customize the "default" metadata forms used by all collections, and also create alternate sets of metadata forms and assign them to specific collections. In creating custom metadata forms, you can choose:

    + + + +

    N.B.The cosmetic and ergonomic details of metadata entry fields remain the same as the fixed metadata pages in previous DSpace releases, and can only be altered by modifying the appropriate stylesheet and JSP pages.

    + +

    All of the custom metadata-entry forms for a DSpace instance are controlled by a single XML file, input-forms.xml, in the config subdirectory under the DSpace home. DSpace comes with a sample configuration that implements the traditional metadata-entry forms, which also serves as a well-documented example. The rest of this section explains how to create your own sets of custom forms.

    + +

    Describing Custom Metadata Forms

    + +

    The description of a set of pages through which submitters enter their metadata is called a form (although it is actually a set of forms, in the HTML sense of the term). A form is identified by a unique symbolic name. In the XML structure, the form is broken down into a series of pages: each of these represents a separate Web page for collecting metadata elements.

    + +

    To set up one of your DSpace collections with customized submission forms, first you make an entry in the form-map. This is effectively a table that relates a collection to a form set, by connecting the collection's Handle to the form name. Collections are identified by handle because their names are mutable and not necessarily unique, while handles are unique and persistent.

    + +

    A special map entry, for the collection handle "default", defines the default form set. It applies to all collections which are not explicitly mentioned in the map. In the example XML this form set is named traditional (for the "traditional" DSpace user interface) but it could be named anything.

    + +

    The Structure of input-forms.xml

    + +

    The XML configuration file has a single top-level element, input-forms, which contains three elements in a specific order. The outline is as follows:

    + 
    +<input-forms>
+
+  <-- Map of Collections to Form Sets -->
+  <form-map>
+    <name-map collection-handle="default" form-name="traditional" />
+    ...
+  </form-map>
+
+  <-- Form Set Definitions -->
+  <form-definitions>
+    <form name="traditional">
+    ...
+  </form-definitions>
+
+  <-- Name/Value Pairs used within Multiple Choice Widgets -->
+  <form-value-pairs>
+    <value-pairs value-pairs-name="common_iso_languages" dc-term="language_iso">
+    ...
+  </form-value-pairs>
+</input-forms>
+
    + +

    Adding a Collection Map

    + +

    Each name-map element within form-map associates a collection with the name of a form set. Its collection-handle attribute is the Handle of the collection, and its form-name attribute is the form set name, which must match the name attribute of a form element.

    + +

    For example, the following fragment shows how the collection with handle "12345.6789/42" is attached to the "TechRpt" form set:

    + 
    +  <form-map>
+    <name-map collection-handle="12345.6789/42" form-name="TechRpt" />
+    ...
+  </form-map>
+
+  <form-definitions>
+    <form name="TechRept">
+    ...
+  </form-definitions>
+
    + +

    It's a good idea to keep the definition of the default name-map from the example input-forms.xml so there is always a default for collections which do not have a custom form set.

    + +

    Getting A Collection's Handle

    + +

    You will need the handle of a collection in order to assign it a custom form set. To discover the handle, go to the "Communities & Collections" page under "Browse" in the left-hand menu on your DSpace home page. Then, find the link to your collection. It should look something like:

    + 
    +    http://myhost.my.edu/dspace/handle/12345.6789/42
+
    + +

    The underlined part of the URL is the handle. It should look familiar to any DSpace administrator. That is what goes in the collection-handle attribute of your name-map element.

    + +

    Adding a Form Set

    + +

    You can add a new form set by creating a new form element within the form-definitions element. It has one attribute, name, which as seen above must match the value of the name-map for the collections it is to be used for.

    + +

    Forms and Pages

    + +

    The content of the form is a sequence of page elements. Each of these corresponds to a Web page of forms for entering metadata elements, presented in sequence between the initial "Describe" page and the final "Verify" page (which presents a summary of all the metadata collected).

    + +

    A form must contain at least one and at most six pages. They are presented in the order they appear in the XML. Each page element must include a number attribute, that should be its sequence number, e.g.

    + 
    +<page number="1">
+
    + +

    The page element, in turn, contains a sequence of field elements. Each field defines an interactive dialog where the submitter enters one of the Dublin Core metadata items.

    + +

    Composition of a Field

    + +

    Each field contains the following elements, in the order indicated. The required sub-elements are so marked:

    + +
    +
    dc-element (Required)
    + +
    Name of the Dublin Core element entered in this field, e.g. contributor.
    + +
    dc-qualifier
    + +
    Qualifier of the Dublin Core element entered in this field, e.g. when the field is contributor.advisor the value of this element would be advisor. Leaving this out means the input is for an unqualified DC element.
    + +
    repeatable
    + +
    Value is true when multiple values of this field are allowed, false otherwise. When you mark a field repeatable, the UI servlet will add a control to let the user ask for more fields to enter additional values. Intended to be used for arbitrarily-repeating fields such as subject keywords, when it is impossible to know in advance how many input boxes to provide.
    + +
    label (Required)
    + +
    Text to display as the label of this field, describing what to enter, e.g. "Your Advisor's Name".
    + +
    input-type (Required)
    + +
    + Defines the kind of interactive widget to put in the form to collect the Dublin Core value. Content must be one of the following keywords: + + +
    + +
    hint (Required)
    + +
    Content is the text that will appear as a "hint", or instructions, next to the input fields. Can be left empty, but it must be present.
    + +
    required
    + +
    When this element is included with any content, it marks the field as a required input. If the user tries to leave the page without entering a value for this field, that text is displayed as a warning message. For example,
    + <required>You must enter a title.</required>
    + Note that leaving the     required element empty will not mark a field as required, e.g.:
    + <required></required>
    +
    + +

    Look at the example input-forms.xml and experiment with a a trial custom form to learn this specification language thoroughly. It is a very simple way to express the layout of data-entry forms, but the only way to learn all its subtleties is to use it.

    + +

    Automatically Elided Fields

    + +

    You may notice that some fields are automatically skipped when a custom form page is displayed, depending on the kind of item being submitted. This is because the DSpace user-interface engine skips Dublin Core fields which are not needed, according to the initial description of the item. For example, if the user indicates there are no alternate titles on the first "Describe" page (the one with a few checkboxes), the input for the title.alternative DC element is automatically elided, even on custom submission pages.

    When a user initiates a submission, DSpace first displays what we'll call the "initial-questions page". By default, it contains three questions with check-boxes: + +
      +
    1. The item has more than one title, e.g. a translated title
      + Controls title.alternative field.
      2. + +
    2. + The item has been published or publicly distributed before
      + Controls DC fields: + + +
      3. + +
    3. The item consists of more than one file
      + Does not affect any metadata input fields.
      4. +
    The answers to the first two questions control whether inputs for certain of the DC metadata fields will displayed, even if they are defined as fields in a custom page. + +

    Conversely, if the metadata fields controlled by a checkbox are not mentioned in the custom form, the checkbox is elided from the initial page to avoid confusing or misleading the user.

    + +

    The two relevant checkbox entries are "The item has more than one title, e.g. a translated title", and "The item has been published or publicly distributed before". The checkbox for multiple titles trigger the display of the field with dc-element equal to 'title' and dc-qualifier equal to 'alternative'. If the controlling collection's form set does not contain this field, then the multiple titles question will not appear on the initial questions page.

    + +

    Adding Value-Pairs

    Finally, your custom form description needs to define the "value pairs" for any fields with input types that refer to them. Do this by adding a value-pairs element to the contents of form-value-pairs. It has the following required attributes: + + Each value-pairs element contains a sequence of pair sub-elements, each of which in turn contains two elements: + + + +

    Unlike the HTML select tag, there is no way to indicate one of the entries should be the default, so the first entry is always the default choice.

    + +

    Example

    + +

    Here is a menu of types of common identifiers:

    + 
    +   <value-pairs value-pairs-name="common_identifiers" dc-term="identifier">
+     <pair>
+       <displayed-value>Gov't Doc #</displayed-value>
+       <stored-value>govdoc</stored-value>
+     </pair>
+     <pair>
+       <displayed-value>URI</displayed-value>
+       <stored-value>uri</stored-value>
+     </pair>
+     <pair>
+       <displayed-value>ISBN</displayed-value>
+       <stored-value>isbn</stored-value>
+     </pair>
+   </value-pairs>
+
    It generates the following HTML, which results in the menu widget below. (Note that there is no way to indicate a default choice in the custom input XML, so it cannot generate the HTML SELECTED attribute to mark one of the options as a pre-selected default.) + 
    +<select name="identifier_qualifier_0">
+<option VALUE="govdoc">Gov't Doc #</option>
+<option VALUE="uri">URI</option>
+<option VALUE="isbn">ISBN</option>
+</select>
+
    + +
    + Identifiers: +
    + +

    Deploying Your Custom Forms

    The DSpace web application only reads your custom form definitions when it starts up, so it is important to remember: + +
    + You must always restart Tomcat (or whatever servlet container you are using) for changes made to the input-forms.xml file take effect. +
    + +

    Any mistake in the syntax or semantics of the form definitions, such as poorly formed XML or a reference to a nonexistent field name, will cause a fatal error in the DSpace UI. The exception message (at the top of the stack trace in the dspace.log file) usually has a concise and helpful explanation of what went wrong. Don't forget to stop and restart the servlet container before testing your fix to a bug.

    + +
    + +
    + Copyright © 2002-2005 MIT and Hewlett Packard +
    + + diff --git a/dspace/docs/update.html b/dspace/docs/update.html new file mode 100644 index 0000000000..4f77704630 --- /dev/null +++ b/dspace/docs/update.html @@ -0,0 +1,522 @@ + + + + DSpace System Documentation: Updating a DSpace Installation + + + + +

    DSpace System Documentation: Updating a DSpace Installation

    + +

    Back to contents

    + +

    This section describes how to update a DSpace installation from one version to the next. Details of the differences between the functionality of each version are given in the Version History section.

    + +

    Updating From 1.2.x to 1.3

    + + FIXME: this is the start of the 1.3 update documentation + +

    In the notes below [dspace] refers to the install directory for your existing DSpace installation, and [dspace-1.3-source] to the source directory for DSpace 1.3. Whenever you see these path references, be sure to replace them with the actual path names on your local system.

    + +
      +

    1. Step one is, of course, to back up all your data before proceeding!! Include all of the contents of [dspace] and the PostgreSQL database in your backup.

      2. +

    2. Get the new DSpace 1.3 source code from the DSpace page on SourceForge and unpack it somewhere. Do not unpack it on top of your existing installation!!

      3. +

    3. Copy the PostgreSQL driver JAR to the source tree. For example:

      +

      cd [dspace]/lib

      +

      cp postgresql.jar [dspace-1.2.2-source]/lib

      +

    4. Take down Tomcat (or whichever servlet container you're using).

      5. +

    5. Install the new config files by moving dstat.cfg and dstat.map from [dspace-1.3-source]/config/ to [dspace]/config

      +

    6. You need to add new parameters to your [dspace]/dspace.cfg:

      + 
      +##### SRB File Storage #####
+
+# The same 'assetstore.incoming' property is used to support the use of SRB 
+# (Storage Resource Broker - see http://www.sdsc.edu/srb/) as an _optional_ 
+# replacement of or supplement to conventional file storage. DSpace will work
+# with or without SRB and full backward compatibility is maintained.
+#
+# The 'assetstore.incoming' property is an integer that references where _new_
+# bitstreams will be stored.  The default (say the starting reference) is zero.
+# The value will be used to identify the storage where all new bitstreams will
+# be stored until this number is changed.  This number is stored in the 
+# Bitstream table (store_number column) in the DSpace database, so older 
+# bitstreams that may have been stored when 'asset.incoming' had a different 
+# value can be found.
+#
+# In the simple case in which DSpace uses local (or mounted) storage the
+# number can refer to different directories (or partitions).  This gives DSpace
+# some level of scalability.  The number links to another set of properties
+# 'assetstore.dir', 'assetstore.dir.1' (remember zero is default), 
+# 'assetstore.dir.2', etc., where the values are directories.
+#
+# To support the use of SRB DSpace uses this same scheme but broadened to 
+# support:
+# - using SRB instead of the local filesystem
+# - using the local filesystem (native DSpace)
+# - using a mix of SRB and local filesystem
+#
+# In this broadened use the 'asset.incoming' integer will refer one of the 
+# following storage locations
+# - a local filesystem directory (native DSpace)
+# - a set of SRB account parameters (host, port, zone, domain, username, 
+#	password, home directory, and resource)
+#
+# Should the be any conflict, like '2' refering to a local directory and 
+# to a set of SRB parameters, the program will select the local directory.
+#
+# If SRB is chosen from the first install of DSpace, it is suggested that
+# 'assetstore.dir' (no integer appended) be retained to reference a local
+# directory (as above under File Storage) because build.xml uses this value 
+# to do a mkdir. In this case, 'assetstore.incoming' can be set to 1 (i.e.
+# uncomment the line in File Storage above) and the 'assetstore.dir' will not 
+# be used.
+#
+# Here is an example set of SRB parameters:
+# Assetstore 1 - SRB
+#srb.host.1 = mysrbmcathost.myu.edu
+#srb.port.1 = 5544
+#srb.mcatzone.1 = mysrbzone
+#srb.mdasdomainname.1 = mysrbdomain
+#srb.defaultstorageresource.1 = mydefaultsrbresource
+#srb.username.1 = mysrbuser
+#srb.password.1 = mysrbpassword
+#srb.homedirectory.1 = /mysrbzone/home/mysrbuser.mysrbdomain
+#srb.parentdir.1 = mysrbdspaceassetstore
+#
+# Assetstore n, n+1, ...
+# Follow same pattern as for assetstores above (local or SRB)
+
+# Directory for history serializations
+history.dir = /dspace/history
+
+# Where to put search index files
+search.dir = /dspace/search
+# Higher values of search.max-clauses will enable prefix searches to work on large 
+# repositories
+#search.max-clauses=2048
+
+# Where to put the logs
+log.dir = /dspace/log
+
+# Where to temporarily store uploaded files
+upload.temp.dir = /dspace/upload
+
+# Maximum size of uploaded files in bytes, must be positive
+# 512Mb
+upload.max = 536870912
+
+
+###### Statistical Report Configuration Settings ######
+
+# directory where live reports are stored
+report.directory = /dspace/reports/
+
      +

    7. Build and install the updated DSpace 1.3 code. Go to the [dspace-1.3-source] directory, and run:

      +

      ant -Dconfig=[dspace]/config/dspace.cfg update

      +

    8. You'll need to make some changes to the database schema in your PostgreSQL database. [dspace-1.3-source]/etc/database_schema_12-13.sql contains the SQL commands to achieve this. If you've modified the schema locally, you may need to check over this and make alterations.

      +

      To apply the changes, go to the source directory, and run:

      +

      psql -f etc/database_schema_12-13.sql [DSpace database name] -h localhost

      +

    9. Customise the stat generating statistics as per the instructions in System Statistical Reports

      +

    10. Initialise the statistics using:

      +

      [dspace]/bin/stat-initial

      +

      [dspace]/bin/stat-general

      +

      [dspace]/bin/stat-report-initial

      +

      [dspace]/bin/stat-report-general

      +

    11. Rebuild the search indices:

      +

      [dspace]/bin/index-all

      +

    12. Copy the .war Web application files in [dspace-1.3-source]/build to the webapps sub-directory of your servlet container (e.g. Tomcat). e.g.:

      +

      cp [dspace-1.3-source]/build/*.war [tomcat]/webapps +

    13. Restart Tomcat.

      14. +
    + +

    Updating From 1.2.1 to 1.2.2

    + +

    The changes in 1.2.2 are only code and config changes so the update should be fairly simple.

    + +

    In the notes below [dspace] refers to the install directory for your existing DSpace installation, and [dspace-1.2.2-source] to the source directory for DSpace 1.2.2. Whenever you see these path references, be sure to replace them with the actual path names on your local system.

    + +
      +

    1. Get the new DSpace 1.2.2 source code from the DSpace page on SourceForge and unpack it somewhere. Do not unpack it on top of your existing installation!!

      2. + +

    2. Copy the PostgreSQL driver JAR to the source tree. For example:

      + + 
      cd [dspace]/lib
+cp postgresql.jar [dspace-1.2.2-source]/lib
      3. + +

    3. Take down Tomcat (or whichever servlet container you're using).

      4. + +

    4. Your 'localized' JSPs (those in jsp/local) now need to be maintained in the source directory. If you have locally modified JSPs in your [dspace]/jsp/local directory, you might like to merge the changes in the new 1.2.2 versions into your locally modified ones. You can use the diff command to compare the 1.2.1 and 1.2.2 versions to do this. Also see the version history for a list of modified JSPs.

      5. + +
    5. +

      You need to add a new parameter to your [dspace]/dspace.cfg for configurable fulltext indexing

      + + +
      ##### Fulltext Indexing settings #####
+# Maximum number of terms indexed for a single field in Lucene.
+# Default is 10,000 words - often not enough for full-text indexing.
+# If you change this, you'll need to re-index for the change
+# to take effect on previously added items.
+# -1 = unlimited (Integer.MAX_VALUE)
+search.maxfieldlength = 10000
+
      6. + +

    6. In [dspace-1.2.2-source] run:

      + + 
      ant -Dconfig=[dspace]/config/dspace.cfg update
      7. + +

    7. Copy the .war Web application files in [dspace-1.2.2-source]/build to the webapps sub-directory of your servlet container (e.g. Tomcat). e.g.:

      + + 
      cp [dspace-1.2.2-source]/build/*.war [tomcat]/webapps
      + +

      If you're using Tomcat, you need to delete the directories corresponding to the old .war files. For example, if dspace.war is installed in [tomcat]/webapps/dspace.war, you should delete the [tomcat]/webapps/dspace directory. Otherwise, Tomcat will continue to use the old code in that directory.

      8. + +

    8. To finialise the install of the new configurable submission forms you need to copy the file [dspace-1.2.2-source]/config/input-forms.xml into [dspace]/config.

      9. + +

    9. Restart Tomcat.

      10. +
    + + +

    Updating From 1.2 to 1.2.1

    + +

    The changes in 1.2.1 are only code changes so the update should be fairly simple.

    + +

    In the notes below [dspace] refers to the install directory for your existing DSpace installation, and [dspace-1.2.1-source] to the source directory for DSpace 1.2.1. Whenever you see these path references, be sure to replace them with the actual path names on your local system.

    + +
      +

    1. Get the new DSpace 1.2.1 source code from the DSpace page on SourceForge and unpack it somewhere. Do not unpack it on top of your existing installation!!

      2. + +

    2. Copy the PostgreSQL driver JAR to the source tree. For example:

      + + 
      cd [dspace]/lib
+cp postgresql.jar [dspace-1.2.1-source]/lib
      3. + +

    3. Take down Tomcat (or whichever servlet container you're using).

      4. + +

    4. Your 'localized' JSPs (those in jsp/local) now need to be maintained in the source directory. If you have locally modified JSPs in your [dspace]/jsp/local directory, you might like to merge the changes in the new 1.2.1 versions into your locally modified ones. You can use the diff command to compare the 1.2 and 1.2.1 versions to do this. Also see the version history for a list of modified JSPs.

      5. + +
    5. +

      You need to add a few new parameters to your [dspace]/dspace.cfg for browse/search and item thumbnails display, and for configurable DC metadata fields to be indexed.

      + + +
      # whether to display thumbnails on browse and search results pages (1.2+)
+webui.browse.thumbnail.show = false
+
+# max dimensions of the browse/search thumbs. Must be <= thumbnail.maxwidth
+# and thumbnail.maxheight. Only need to be set if required to be smaller than
+# dimension of thumbnails generated by mediafilter (1.2+)
+#webui.browse.thumbnail.maxheight = 80
+#webui.browse.thumbnail.maxwidth = 80
+
+# whether to display the thumb against each bitstream (1.2+)
+webui.item.thumbnail.show = true
+
+# where should clicking on a thumbnail from browse/search take the user
+# Only values currently supported are "item" and "bitstream"
+#webui.browse.thumbnail.linkbehaviour = item
+
+
+ ##### Fields to Index for Search #####
+
+# DC metadata elements.qualifiers to be indexed for search
+# format: - search.index.[number] = [search field]:element.qualifier
+#         - * used as wildcard
+
+###      changing these will change your search results,     ###
+###  but will NOT automatically change your search displays  ###
+
+search.index.1 = author:contributor.*
+search.index.2 = author:creator.*
+search.index.3 = title:title.*
+search.index.4 = keyword:subject.*
+search.index.5 = abstract:description.abstract
+search.index.6 = author:description.statementofresponsibility
+search.index.7 = series:relation.ispartofseries
+search.index.8 = abstract:description.tableofcontents
+search.index.9 = mime:format.mimetype
+search.index.10 = sponsor:description.sponsorship
+search.index.11 = id:identifier.*
      6. + +

    6. In [dspace-1.2.1-source] run:

      + + 
      ant -Dconfig=[dspace]/config/dspace.cfg update
      7. + +

    7. Copy the .war Web application files in [dspace-1.2.1-source]/build to the webapps sub-directory of your servlet container (e.g. Tomcat). e.g.:

      + + 
      cp [dspace-1.2.1-source]/build/*.war [tomcat]/webapps
      + +

      If you're using Tomcat, you need to delete the directories corresponding to the old .war files. For example, if dspace.war is installed in [tomcat]/webapps/dspace.war, you should delete the [tomcat]/webapps/dspace directory. Otherwise, Tomcat will continue to use the old code in that directory.

      8. + +

    8. Restart Tomcat.

      9. +
    +

    Updating From 1.1 (or 1.1.1) to 1.2

    + +

    The process for upgrading to 1.2 from either 1.1 or 1.1.1 is the same. If you are running DSpace 1.0 or 1.0.1, you need to follow the instructions for upgrading from 1.0.1 to 1.1 to before following these instructions.

    + +

    Note also that if you've substantially modified DSpace, these instructions apply to an unmodified 1.1.1 DSpace instance, and you'll need to adapt the process to any modifications you've made.

    + +

    This document refers to the install directory for your existing DSpace installation as [dspace], and to the source directory for + DSpace 1.2 as [dspace-1.2-source]. Whenever you see these path references below, be sure to replace them with the actual path names on your local system. + +

      +

    1. Step one is, of course, to back up all your data before proceeding!! Include all of the contents of [dspace] and the PostgreSQL database in your backup.

      2. + +

    2. Get the new DSpace 1.2 source code from the DSpace page on SourceForge and unpack it somewhere. Do not unpack it on top of your existing installation!!

      3. + +

    3. Copy the required Java libraries that we couldn't include in the bundle to the source tree. For example:

      + + 
      cd [dspace]/lib
+cp activation.jar servlet.jar mail.jar [dspace-1.2-source]/lib
      4. + +

    4. Stop Tomcat (or other servlet container.)

      5. + +
    5. +

      It's a good idea to upgrade all of the various third-party tools that DSpace uses to their latest versions:

      + +
      6. + +
    6. +

      You need to add the following new parameters to your [dspace]/dspace.cfg:

      + + 
      ##### Media Filter settings #####
+# maximum width and height of generated thumbnails
+thumbnail.maxwidth  80
+thumbnail.maxheight 80
      + +

      There are one or two other, optional extra parameters (for controlling the pool of database connections). See the version history for details. If you leave them out, defaults will be used.

      + +

      Also, to avoid future confusion, you might like to remove the following property, which is no longer required:

      + + 
      config.template.oai-web.xml = [dspace]/oai/WEB-INF/web.xml
      +
      7. + +

    7. The layout of the installation directory (i.e. the structure of the contents of [dspace]) has changed somewhat since 1.1.1. First up, your 'localized' JSPs (those in jsp/local) now need to be maintained in the source directory. So make a copy of them now!

      + +

      Once you've done that, you can remove [dspace]/jsp and [dspace]/oai, these are no longer used. (.war Web application archive files are used instead).

      + +

      Also, if you're using the same version of Tomcat as before, you need to remove the lines from Tomcat's conf/server.xml file that enable symbolic links for DSpace. These are the <Context> elements you added to get DSpace 1.1.1 working, looking something like this:

      + + 
      <Context path="/dspace" docBase="dspace" debug="0" reloadable="true" crossContext="true">
+  <Resources className="org.apache.naming.resources.FileDirContext" allowLinking="true" />
+</Context>
      + +

      Be sure to remove the <Context> elements for both the Web UI and the OAI Web applications.

      +
      8. + +

    8. Build and install the updated DSpace 1.2 code. Go to the DSpace 1.2 source directory, and run:

      + + 
      ant -Dconfig=[dspace]/config/dspace.cfg update
      9. + +

    9. Copy the new config files in config to your installation, e.g.:

      + + 
      cp [dspace-1.2-source]/config/news-* [dspace-1.2-source]/config/mediafilter.cfg [dspace-1.2-source]/config/dc2mods.cfg [dspace]/config
      10. + +
    10. +

      You'll need to make some changes to the database schema in your PostgreSQL database. [dspace-1.2-source]/etc/database_schema_11-12.sql contains the SQL commands to achieve this. If you've modified the schema locally, you may need to check over this and make alterations.

      + +

      To apply the changes, go to the source directory, and run:

      + + 
      psql -f etc/database_schema_11-12.sql [DSpace database name] -h localhost
      +
      11. + +

    11. A tool supplied with the DSpace 1.2 codebase will then update the actual data in the relational database. Run it using:

      + + 
      [dspace]/bin/dsrun org.dspace.administer.Upgrade11To12
      12. + +

    12. Then rebuild the search indices:

      + + 
      [dspace]/bin/index-all
      13. + +

    13. Delete the existing symlinks from your servlet container's (e.g. Tomcat's) webapp sub-directory.

      + +

      Copy the .war Web application files in [dspace-1.2-source]/build to the webapps sub-directory of your servlet container (e.g. Tomcat). e.g.:

      + + 
      cp [dspace-1.2-source]/build/*.war [tomcat]/webapps
      14. + +

    14. Restart Tomcat.

      15. + +

    15. To get image thumbnails generated and full-text extracted for indexing automatically, you need to set up a 'cron' job, for example one like this:

      + + 
      # Run the media filter at 02:00 every day
+0 2 * * * [dspace]/bin/filter-media
      + +

      You might also wish to run it now to generate thumbnails and index full text for the content already in your system.

      16. + +
    16. +

      Note 1: This update process has effectively 'touched' all of your items. Although the dates in the Dublin Core metadata won't have changed (accession date and so forth), the 'last modified' date in the database for each will have been changed.

      + +

      This means the e-mail subscription tool may be confused, thinking that all items in the archive have been deposited that day, and could thus send a rather long email to lots of subscribers. So, it is recommended that you turn off the e-mail subscription feature for the next day, by commenting out the relevant line in DSpace's cron job, and then re-activating it the next day.

      + +

      Say you performed the update on 08-June-2004 (UTC), and your e-mail subscription cron job runs at 4am (UTC). When the subscription tool runs at 4am on 09-June-2004, it will find that everything in the system has a modification date in 08-June-2004, and accordingly send out huge emails. So, immediately after the update, you would edit DSpace's 'crontab' and comment out the /dspace/bin/subs-daily line. Then, after 4am on 09-June-2004 you'd 'un-comment' it out, so that things proceed normally.

      + +

      Of course this means, any real new deposits on 08-June-2004 won't get e-mailed, however if you're updating the system it's likely to be down for some time so this shouldn't be a big problem.

      +
      17. + +
    17. +

      Note 2: After consulation with the OAI community, various OAI-PMH changes have occurred:

      + + + +

      The above means that, if already registered and harvested, you will need to re-register your repository, effectively as a 'new' OAI-PMH data provider. You should also consider posting an announcement to the OAI implementers e-mail list so that harvesters know to update their systems.

      + +

      Also note that your site may, over the next few days, take quite a big hit from OAI-PMH harvesters. The resumption token support should alleviate this a little, but you might want to temporarily whack up the database connection pool parameters in [dspace]/config/dspace.cfg. See the dspace.cfg distributed with the source code to see what these parameters are and how to use them. (You need to stop and restart Tomcat after changing them.)

      + +

      I realize this is not ideal; for discussion as to the reasons behind this please see relevant posts to the OAI community: post one, post two, as well as this post to the dspace-tech mailing list.

      + +

      If you really can't live with updating the base URL like this, you can fairly easily have thing proceed more-or-less as they are, by doing the following:

      + + + +

      However, note that in this case, all the records will be re-harvested by harvesters anyway, so you still need to brace for the associated DB activity; also note that the set spec changes may not be picked up by some harvesters. It's recommended you read the above-linked mailing list posts to understand why the change was made.

      +
      18. +
    + +

    Now, you should be finished!

    + + +

    Updating From 1.1 to 1.1.1

    + +

    Fortunately the changes in 1.1.1 are only code changes so the update is fairly simple.

    + +

    In the notes below [dspace] refers to the install directory for your existing DSpace installation, + and [dspace-1.1.1-source] to the source directory for DSpace 1.1.1. Whenever you see these path + references, be sure to replace them with the actual path names on your local system.

    + +
      +

    1. Take down Tomcat.

      2. + +

    2. It would be a good idea to update any of the third-party tools used by DSpace at this point (e.g. PostgreSQL), following the instructions provided with the relevant tools.

      3. + +

    3. In [dspace-1.1.1-source] run:

      + + 
      ant -Dconfig=[dspace]/config/dspace.cfg update
      4. + +

    4. If you have locally modified JSPs of the following JSPs in your [dspace]/jsp/local directory, you might like to merge the changes in the new 1.1.1 versions into your locally modified ones. You can use the diff command to compare the 1.1 and 1.1.1 versions to do this. The changes are quite minor.

      + + 
      collection-home.jsp
+admin/authorize-collection-edit.jsp
+admin/authorize-community-edit.jsp
+admin/authorize-item-edit.jsp
+admin/eperson-edit.jsp
      5. + +

    5. Restart Tomcat.

      6. +
    + +

    Updating From 1.0.1 to 1.1

    + +

    To upgrade from DSpace 1.0.1 to 1.1, follow the steps below. Your dspace.cfg does not need to be changed. + In the notes below [dspace] refers to the install directory for your existing DSpace installation, + and [dspace-1.1-source] to the source directory for DSpace 1.1. Whenever you see these path + references, be sure to replace them with the actual path names on your local system.

    + +
      +

    1. Take down Tomcat (or whichever servlet container you're using).

      2. + +

    2. We recommend that you upgrage to the latest version of PostgreSQL (7.3.2). Included are some notes to help you do this. Note you will also have to upgrade Ant to version 1.5 if you do this.

      3. + +

    3. Make the necessary changes to the DSpace database. These include a couple of minor schema changes, and some new indices which should improve performance. Also, the names of a couple of database views have been changed since the old names were so long they were causing problems. First run psql to access your database (e.g. psql -U dspace -W and then enter the password), and enter these SQL commands:

      + + 
      ALTER TABLE bitstream ADD store_number INTEGER;
+UPDATE bitstream SET store_number = 0;
+
+ALTER TABLE item ADD last_modified TIMESTAMP;
+CREATE INDEX last_modified_idx ON Item(last_modified);
+
+CREATE INDEX eperson_email_idx ON EPerson(email);
+CREATE INDEX item2bundle_item_idx on Item2Bundle(item_id);
+REATE INDEX bundle2bitstream_bundle_idx ON Bundle2Bitstream(bundle_id);
+CREATE INDEX dcvalue_item_idx on DCValue(item_id);
+CREATE INDEX collection2item_collection_idx ON Collection2Item(collection_id);
+CREATE INDEX resourcepolicy_type_id_idx ON ResourcePolicy (resource_type_id,resource_id);
+CREATE INDEX epersongroup2eperson_group_idx on EPersonGroup2EPerson(eperson_group_id);
+CREATE INDEX handle_handle_idx ON Handle(handle);
+CREATE INDEX sort_author_idx on ItemsByAuthor(sort_author);
+CREATE INDEX sort_title_idx on ItemsByTitle(sort_title);
+CREATE INDEX date_issued_idx on ItemsByDate(date_issued);
+
+DROP VIEW CollectionItemsByDateAccessioned;
+
+DROP VIEW CommunityItemsByDateAccessioned;
+CREATE VIEW CommunityItemsByDateAccession as SELECT Community2Item.community_id, ItemsByDateAccessioned.*  FROM ItemsByDateAccessioned, Community2Item WHERE ItemsByDateAccessioned.item_id = Community2Item.item_id;
+CREATE VIEW CollectionItemsByDateAccession AS SELECT collection2item.collection_id, itemsbydateaccessioned.items_by_date_accessioned_id, itemsbydateaccessioned.item_id, itemsbydateaccessioned.date_accessioned FROM itemsbydateaccessioned, collection2item WHERE (itemsbydateaccessioned.item_id = collection2item.item_id);
      4. + +

    4. Fix your JSPs for Unicode. If you've modified the site 'skin' (jsp/local/layout/header-default.jsp) you'll need to add the Unicode header, i.e.:

      + + 
      <meta http-equiv="Content-Type" content="text/html; charset=UTF-8">
      + +

      to the <HEAD> element. If you have any locally-edited JSPs, you need to add this page directive to the top of all of them:

      + + 
      <%@ page contentType="text/html;charset=UTF-8" %>
      + +

      (If you haven't modified any JSPs, you don't have to do anything.)

      5. + + +

    5. Copy the required Java libraries that we couldn't include in the bundle to the source tree. For example:

      + + 
      cd [dspace]/lib
+cp *.policy activation.jar servlet.jar mail.jar [dspace-1.1-source]/lib
      6. + + +

    6. Compile up the new DSpace code, replacing [dspace]/config/dspace.cfg with the path to your current, LIVE configuration. (The second line, touch `find .`, is a precaution, which ensures that the new code has a current datestamp and will overwrite the old code. Note that those are back quotes.)

      + + 
      cd [dspace-1.1-source]
+touch `find .`
+ant
+ant -Dconfig=[dspace]/config/dspace.cfg update
      7. + + +

    7. Update the database tables using the upgrader tool, which sets up the new >last_modified date in the item table:

      + + 
      Run [dspace]/bin/dsrun org.dspace.administer.Upgrade101To11
      8. + + +

    8. Run the collection default authorisation policy tool:

      + + 
      [dspace]/bin/dsrun org.dspace.authorize.FixDefaultPolicies
      9. + + +

    9. Fix the OAICat properties file. Edit [dspace]/config/templates/oaicat.properties. Change the line that says

      + + 
      Identify.deletedRecord=yes
      + +

      To:

      + + 
      Identify.deletedRecord=persistent
      + +

      This is needed to fix the OAI-PMH 'Identity' verb response. Then run [dspace]/bin/install-configs.

      10. + + +

    10. Re-run the indexing to index abstracts and fill out the renamed database views:

      + + 
      [dspace]/bin/index-all
      + + +

    11. Restart Tomcat. Tomcat should be run with the following environment variable set, to ensure that Unicode is handled properly. Also, the default JVM memory heap sizes are rather small. Adjust -Xmx512M (512Mb maximum heap size) and -Xms64M (64Mb Java thread stack size) to suit your hardware.

      + + 
      JAVA_OPTS="-Xmx512M -Xms64M -Dfile.encoding=UTF-8"
      12. +
    + + +
    + +
    + Copyright © 2002-2004 MIT and Hewlett Packard +
    + + diff --git a/dspace/make-release-package b/dspace/make-release-package index 6954dd4da1..33ebd01c35 100755 --- a/dspace/make-release-package +++ b/dspace/make-release-package @@ -1,18 +1,10 @@ #!/bin/sh -USAGE="$0 [-d ] cvs-tag version" +USAGE="$0 cvs-tag version" # Just in case you need to 'socksify' etc CVS_COMMAND="cvs" -DOC_CVSTAG="no" - -# Check for doc CVS tag -if [ "$1" = "-d" ]; then - DOC_CVSTAG=$2 - shift;shift -fi - # Check we have required command-line arguments if [ "$#" != "2" ]; then echo $USAGE @@ -21,7 +13,6 @@ fi FILENAME="dspace-$2-source" - mkdir tmp cd tmp @@ -34,21 +25,6 @@ rm -f dspace/make-release-package # Or silly cvsignore files rm -f `find dspace -name .cvsignore` -# Check out docs if appropriate -if [ "$DOC_CVSTAG" != "no" ]; then - - echo "Checking out docs..." - cd dspace - $CVS_COMMAND -Q export -r $DOC_CVSTAG docs - - # Remove unwanted stuff - rm -f docs/.cvsignore - rm -rf docs/originals - rm -f docs/make-doc-package - - cd .. -fi - echo "Creating tarball..." mv dspace $FILENAME @@ -60,4 +36,4 @@ cd .. mv tmp/$FILENAME.tar.gz . rm -r tmp -echo "Package created as $FILENAME.tar.gz" +echo "Package created as $FILENAME.tar.gz" \ No newline at end of file