Data Archiving - More Technical

Data Archiving

A live R/3 system will, over time, accumulate massive amounts of data. The increasing volume of data can degrade system performance, causing longer transaction processing times. System maintenance tasks take longer to complete, and batch runs take longer to process.
 

Does this sound familiar?

The solution to these problems is data archiving, the removal and backup of R/3 data from the online database. Deletion of data from the database should not be considered, as there may be legal, regulatory, and business considerations. Some of the data may be required at a later time to resolve customer service problems, generate reports, or to facilitate data analysis. Therefore, the data must be preserved in a manner that allows users to quickly access the information if needed.

SAP Data Archiving is the only SAP-supported method for removing data from the database consistently and maintaining the data availability (read access) for future business requirements.


SAP data archiving:

Prevents and resolves performance problems caused by large volumes of data (not all of the data needs to reside online)

Makes use of hardware, such as disk and memory, more efficiently

Reduce the cost of the system maintenance (for example, backup, recovery, and upgrades)

Ensures compliance with statutory data retention rules

Ensures that data can be reused at a later date


Note: Data archiving reduces the amount of data in the database, but does not affect the size allotment of the database itself. Data archiving improves system performance by maintaining system manageability.

WHEN TO START ARCHIVING
 

Data archiving should be addressed during the planning stages of an R/3 implementation. It plays an important role in maintaining the R/3 system's performance levels. However, data archiving should not be viewed as a final solution to prevent system downtime. Data that is nonessential for auditing purpose should be archived as soon as possible.

Data Archiving is an interdisciplinary process, demanding high levels of cooperation between the business user department and the IT department. IT system administration tries to keep the database small by deleting unnecessary data from the database. However, the user community wants access to all documents online. To meet the needs of both communities, the data archiving planning process must reach a compromise.


Data archiving must be planned centrally for all departments. This includes considerations relating to:


The required system size in relation to the anticipated data volume (sizing)
The required residence times (the period that the data is required online)
You should ensure that you have met all foreseeable auditing requirements before beginning the data archiving process.


back to top

WHAT TO ARCHIVE

To identify data archival, you must know the business processes involved. Based on the business process information, database table growth can be analyzed and the table association to the specific functions can be sorted. Consequently, the archive object can be identified with the table and business information.

You must know the business processes involved before you identify data for archival. With this knowledge, database table growth can be analyzed to associate specific functions with the tables, simplifying the identification of objects for archival.

Application data should not be archived until:

• It is no longer required for processing

• It requires no further changes

• You are unlikely to need to display it

Before archiving application data, you must ensure that all the required documentation has been created for subsequent auditing.

Information Retention Requirements
Information retention requirements can be divided into the following categories:

Legal requirements: Basic accounting requirements and specific requirements imposed by the tax authorities
Internal business requirements: Requirements imposed by service departments, controlling or internal auditing
Technical requirements: Access times, durability of data media, long-term readability of the information

back to top
 

ARCHIVING PHASES

By using SAP Data Archiving, important business objects such as accounting documents and material master records can be selectively removed from the database in such a way that the user does not have to worry about the underlying physical table design. Data for archiving is stored first in the file system, then transferred to other storage media. For security reasons, archived data is only deleted from the database if the previously created archive files were read successfully.

At the very basic level, data archiving is a three-step procedure:

 

• Create archiving files

• Verify archived data and delete it from database

• Optional, move the archived files to a storage management system

Data in the R/3 Database is archived using archiving objects that describe the data structure and context. For example, financial accounting documents are archived using the archiving object FI_DOCUMNT, which includes the document header, company code-dependent postings, change documents, SAPscript texts, and other elements.




1. Create Archive Files

Using an archiving object, the archiving program writes the data to be archived sequentially from the R/3 database to the archive files. To maintain system consistency, the archiving program ensures that only the data from completed business processed will be archived. Master data can only be archived if it has been marked with a deletion indicator.

2. Delete the Data

After the data to be archived has been written to archiving files, it is removed from the database by the deletion program. For data security reasons, the deletion program removes the data only after the related archive files have been successfully read.

3. Storage

The created archive files can then be moved to a tertiary storage system or copied to tape. The removal to an external storage system can be triggered manually or automatically.


back to top

ARCHIVING ADMINISTRATION

The technical process flow of archiving generally includes three phases:
 

1. Customizing

2. Execution of Archiving

3. Management of Archiving Files

Phase I: Customizing
General Customizing

• Application-specific customizing
  The individual applications provide partial criteria for the archivability or deletability of application data. These settings are made in customizing for the relevant application. An example of this is the setting of the residence time.

• Customizing for SAP ArchiveLink
  Some customizing is required if the archive files are to be transferred to a storage system over the ArchiveLink interface.

• Platform-independent filenames
  The archive files are stored in different directories depending on the operating system. The path and filenames therefore have a different syntax. Consequently, platform-independent "logical" filenames are used so that the archiving programs do not have to handle the technical details of allocating physical names. In the ADK runtime system, the logical filename is used to generate a corresponding physical filename.

Object-specific Customizing

The following parameters must be set in archiving object-specific customizing:
 

• Logical filename
  Select a logical filename for use by the archiving object. At runtime, the ADK generates a physical filename from the logical filename.

• Archive file size
  Enter the maximum size that an archive file is permitted to reach before it is closed. You can either establish the size in MB or the maximum number of data objects that can be written to an archive file. Both values are queried by an OR operation. As soon as either the maximum file size or the maximum number of data objects is reached, the system will close the archive file and create a new file.

• Deletion program settings
  You can specify whether the system should execute the deletion program automatically after an archive file has been created. Using the Commit Counter parameter, the number of data objects required in the deletion program before a DB commit is triggered.

Phase II: Execution of Archiving Session
Archive
The archiving session selects the data objects from the database. The system observes the relevant integrity conditions that characterize a data object. It then checks each archiving object to see whether it can be archived. If this is the case, the data object is written to the archive file. If you have set the deletion program to run automatically in customizing, the accompanying deletion run starts automatically.

Delete
The deletion run must be scheduled separately if it was not executed automatically at the archiving stage. A selection of archive files must be made from which data objects in the current deletion run have been read and subsequently deleted from the database.
For some archiving objects, special preprocessing and post-processing programs prepare and clean up an archiving session, such as post-processing programs that update statistical data after a successful archiving session.

Phase III: Archiving management
The system does more than just write data to archive files. In order to access the files at a later date, R/3 stores management data on the archive files that can be displayed for any archiving session. This includes:
 

• Information on all archive files belonging to the archiving session

• Name, date, time and status of the archiving session

After a certain period of time, archived data may no longer be needed. For example, the data may no longer need to be kept for legal or business reasons, or the data has been transported to another format where it remains accessible (such as print lists, microfiche or statistics systems).

back to top

ARCHIVING OBJECTS
Database tables are not easily assigned to individual application functions, since table contents generally originate from an assortment of functions. To consistently archive the business data, the archiving object, a central element of SAP Data Archiving, is introduced.

The archiving object defines or specifies:

• The smallest unit that can be archived and deleted from the database

• The application objects to be archived

• Which data objects must be accessed and how to access them in order to archive a business object fully

An archiving object comprises three main components:
 

• Data declaration
  This describes all of the relevant data objects that make up an application object.
   

• Archiving programs
  These include:

1. Write program - Writes data objects sequentially to the archive files

2. Deletion program - Deletes data objects from the database that have been successfully written to an archive file

3. Preprocessing program (optional) - Prepares the data objects for the archiving session. Identifies the objects to be archived (sets a deletion indicator)

4. Display program (optional): Allows archived data objects to be read

5. Post-processing program (optional): Performs post-processing after archiving, such as updating statistical data

6. Reload program (optional): Loads data objects back into the database

• Customizing settings
  Sets archiving object-specific parameters for an archiving session

back to top

ARCHIVING DEVELOPMENT KIT

The ADK is the central component for controlling an archiving session. It contains functions for the analysis of individual archiving objects and for the administration of archiving sessions. The ADK includes the following transactions:
 

• Transaction SARA for archive administration

• Transaction AOBJ for the definition of archiving objects

• Transaction ACLA for the definition of archiving classes

• Transaction DB15 for monitoring the storage in the database

back to top

ACCESS ARCHIVED DATA

There are different methods to access archived data, depending on the application area involved and the archiving object used.
 

1. Direct Access
   The user displays archived data directly from his usual transaction. To do this, index information on archived data must be maintained in the database, for example, using the SAP Archiving Information System (SAP AS). The applications must have been modified especially for this. This method is not possible for all archiving objects.

2. SAP Archiving Information System (SAP AS)
   The SAP AS offers generic access to archived data. This access can be enhanced using specially developed intelligent display programs. SAP has already included such enhancements in some archiving objects.

3. Reporting
   Reporting on archived data is also possible, depending on the application, the detail level of the result, and whether archived data can be processed together with data from the database (mixed reporting). In the case of a list-oriented display, the system does not notify the user that the data selection may be incomplete, as some data objects have already been archived.

4. Customized Access
   By using the tools available in the standard system, the Archive Development Kit (ADK) and the SAP Archive Information System (SAP AS), it is possible to modify access to archived data in line with specific customer requirements.

5. Document Relationship Browser (DRB)
   The SAP DRB offers generic access to archived and online data. This access can be enhanced using specially developed intelligent display programs. SAP has already included such enhancements in some archiving objects

back to top

ARCHIVE INFORMATION SYSTEMS

The SAP Archive Information System is a data display and retrieval tool that is fully integrated into the archiving environment, assisting the user with information retrieval from R/3 data archives.

From Release 4.5A, the SAP Archive Information System (SAP AS) is a part of the R/3 standard functionality and can be installed on previous releases, starting with Release 3.0D. For more information, refer to SAPNote 99388.

Data retrieval takes place using archive information structures, transparent database tables that contain archive data. As with other R/3 information systems, such as the Logistics Information Structures (LIS) or the Sales Information System (VIS), these tables are referred to as information structures. These are composed of the structure itself, the relevant transparent database table and an evaluation program. When a new information structure is created, only the structure is created in the first instance. The database table and the evaluation program are only generated when the information structure is activated. An archive information structure is always assigned to an individual archiving object.


To retrieve archived data for an archiving object, at least one archive information structure must exist. The information structure should contain all the fields that are required for the retrieval. The user can, if necessary, change the contents of the information structure by removing or adding fields from an existing SAP field catalog.


When displaying archived data, the user can usually choose between several views (for example, technical view or business view) for displaying data objects. A technical view is available for all archiving objects. The availability of application views depends on the implementation settings for the relevant archiving object. For more information on the SAP Archiving Information System, refer to the SAP Library under CA - Cross-Application Components ®CA - Application Data Archiving® Introduction®Archive Information System (SAP AS).


back to top

ARCHIVING DATA RELOAD

SAP Data Archiving allows data to be read years after it has been archived, and across release upgrades. However, the data must be correctly presented by the external storage system. Therefore, reloading archived data is not generally necessary. For this reason, reload programs are not available for all archiving objects. SAP does not plan to support reload functions for all archiving objects. There are two possible scenarios for the reloading of archived data:

Reloading directly after archiving
Typically, this occurs if you realize that you have archived too much data due to incorrect selection criteria. The reload programs are specifically designed for this situation. Therefore, reloading usually should not cause any problems.

Reloading some time after archiving
The reloading of archived data long after the archiving is problematic from a business point of view, since it can lead to database inconsistencies. For instance, a number range may, in the meantime, have become outdated and then been reissued. If you reload old documents, more recent documents with identical numbers are overwritten. Consequently, before reloading, you should always discuss this with your SAP consultant.


back to top

ARCHIVING TOOLS

Archive Administration (SARA)
Archive Administration is the standard SAP tool for archiving application data. It enables you to archive various kinds of SAP data, from cost center (plan data) to maintenance notifications, task lists, and posting documents.

Archive Information System (SARI)
Archive Information System (AS) is a generic standard tool delivered by SAP that eases the customized accesses to the archived data.


back to top