App Control server collects inventories from all computers that are running App Control Agent, and tracks any inventory changes.
Once a computer is added to the system, it sends its inventory of all “interesting” files to its App Control Server. This can be between 10K and 50K files per endpoint. Once this inventory is sent, only a small number of new files (changes) will be reported on daily bases. This usually includes up to 500 file additions or deletions per endpoint.
Since the App Control Server maintains only the current inventory and only has a limited file deletion/modification history, the database growth is much slower after all endpoints are initialized.
The following table gives more information regarding the growth of specific database files:
| Name | File name | Growth | Growth control options |
|---|---|---|---|
| PRIMARY | das.mdf | The growth of this file is not limited since it contains all unique file metadata ever seen in the environment as well as all rules and approvals. Rapid growth of this file after a steady state has been reached could indicate excessive generation of unique files on endpoints. | Optional deletion of files with zero prevalence can be turned on. See the next section, “Pruning File Catalog”, for more information. |
| SECONDARY | das_events.ndf | Growth is limited by event retention thresholds (time and size) set on the System Configuration page | Lowering the thresholds will limit file growth. |
| ABINST | das_abinst.ndf | Growth is proportional to the number of interesting files on endpoints. | Custom rules can be used to ignore “noisy” files that are not interesting to track. |
| ABTEMP | das_abtemp.ndf | Growth is proportional to the number of interesting files on endpoints. | Custom rules can be used to ignore “noisy” files that are not interesting to track. |
| Log file | das_log.ldf | Largest growth occurs during large transactions, most notably during product upgrades. | Changing the server recovery mode or more frequent transaction log backups. |
| Temp SQL data file | Tempdb.mdf | These files grow mostly during expensive background tasks (like the Daily Prune Task), and should never get larger than 50 GB in total.
|
SQL Server service restart will reset these files to 0 size.
|
| Temp SQL log file | Tempdb.ldf |
Note that SQL Server only grows its data files and will not reduce them, even as underlying data is deleted. Essentially, a file size represents the maximum watermark of the size that was reached at some point. The only exception to this is the log file, which is controlled by the backup and SQL Server recovery modes.
The following MSDN article describes how to shrink the empty data files. Shrinking data file can have a negative performance impact. Use these steps only when a large amount of data is deleted from the database (e.g., if event retention has been reduced), and you don’t expect the database to reuse this space.
http://technet.microsoft.com/en-us/library/ms190757.aspx
Events Growth
The SECONDARY file group (das_events.ndf) will grow based on limits set in the configuration for both public and internal events.
Here is estimated number of daily public and internal events for 1000 agents:
| Agent Count | Daily public events | Daily internal events |
|---|---|---|
| 1 | 250 | 100 |
| 1,000 | 250,000 | 100,000 |
| 100,000 | 25,000,000 | 10,000,000 |
A single public event requires approx. 0.5 KB in database, and an internal event requires 0.8 KB. Based on that, here is estimate on final Events filegroup target sizes for different event limits (configurable):
| Public events | Internal events | Database file size (das_events.ndf) |
|---|---|---|
| 1,000,000 | 100,000 | 0.6 GB |
| 10,000,000 | 1,000,000 | 6 GB |
| 100,000,000 | 10,000,000 | 60 GB |
Default retention limit for public events is 10M, and for internal events is 100,000.
Pruning the File Catalog
File catalog pruning is disabled by default. You can turn it on by changing the shepherd_config property PurgeAntibodiesPeriodDays. A value of 0 means that files will never be deleted from the catalog.
When this pruning is enabled, App Control will automatically delete all files with a prevalence of 0 (i.e., not existing on any endpoint), after N days from reaching 0 prevalence. Note that if a 0-prevalence file reappears on any endpoint during these N days, its pruning will be canceled.
This feature is useful when a App Control environment has a lot of unique temporary files on the endpoints, causing the PRIMARY filegroup to grow unbound. The downside is that the records of these unique files might still provide some forensic value even after the files themselves are gone.
