The table level gc_grace_seconds parameter controls how long Cassandrawill retain tombstones through compaction events before finally removingthem. This duration should directly reflect the amount of time a userexpects to allow before recovering a failed node. Aftergc_grace_seconds has expired the tombstone may be removed (meaningthere will no longer be any record that a certain piece of data wasdeleted), but as a tombstone can live in one sstable and the data itcovers in another, a compaction must also include both sstable for atombstone to be removed. More precisely, to be able to drop an actualtombstone the following needs to be true;
If an sstable contains only tombstones and it is guaranteed that thatsstable is not shadowing data in any other sstable compaction can dropthat sstable. If you see SSTables with only tombstones (note that TTL:eddata is considered tombstones once the time to live has expired) but itis not being dropped by compaction, it is likely that other SSTablescontain older data. There is a tool called sstableexpiredblockers thatwill list which SSTables are droppable and which are blocking them frombeing dropped. This is especially useful for time series compaction withTimeWindowCompactionStrategy (and the deprecatedDateTieredCompactionStrategy). With TimeWindowCompactionStrategy itis possible to remove the guarantee (not check for shadowing data) byenabling unsafe_aggressive_sstable_expiration.
With incremental repairs Cassandra must keep track of what data isrepaired and what data is unrepaired. With anticompaction repaired datais split out into repaired and unrepaired SSTables. To avoid mixing upthe data again separate compaction strategy instances are run on the twosets of data, each instance only knowing about either the repaired orthe unrepaired SSTables. This means that if you only run incrementalrepair once and then never again, you might have very old data in therepaired SSTables that block compaction from dropping tombstones in theunrepaired (probably newer) SSTables.
Since tombstones and data can live in different SSTables it is importantto realize that losing an sstable might lead to data becoming live again- the most common way of losing SSTables is to have a hard drive breakdown. To avoid making data live tombstones and actual data are always inthe same data directory. This way, if a disk is lost, all versions of apartition are lost and no data can get undeleted. To achieve this acompaction strategy instance per data directory is run in addition tothe compaction strategy instances containing repaired/unrepaired data,this means that if you have 4 data directories there will be 8compaction strategy instances running. This has a few more benefits thanjust avoiding data getting undeleted:
When an sstable is written a histogram with the tombstone expiry timesis created and this is used to try to find SSTables with very manytombstones and run single sstable compaction on that sstable in hope ofbeing able to drop tombstones in that sstable. Before starting this itis also checked how likely it is that any tombstones will actually willbe able to be dropped how much this sstable overlaps with otherSSTables. To avoid most of these checks the compaction optionunchecked_tombstone_compaction can be enabled.
Since it might not be possible to drop any tombstones when doing asingle sstable compaction we need to make sure that one sstable is notconstantly getting recompacted - this option states how often weshould try for a given sstable.
The single sstable compaction has quite strict checks for whether itshould be started, this option disables those checks and for someusecases this might be needed. Note that this does not change anythingfor the actual compaction, tombstones are only dropped if it is safeto do so - it might just rewrite an sstable without being able to dropany tombstones.
In geotechnical engineering, soil compaction is the process in which stress applied to a soil causes densification as air is displaced from the pores between the soil grains. When stress is applied that causes densification due to water (or other liquid) being displaced from between the soil grains, then consolidation, not compaction, has occurred. Normally, compaction is the result of heavy machinery compressing the soil, but it can also occur due to the passage of, for example, animal feet.
Soil compaction is a vital part of the construction process. It is used for support of structural entities such as building foundations, roadways, walkways, and earth retaining structures to name a few. For a given soil type certain properties may deem it more or less desirable to perform adequately for a particular circumstance. In general, the preselected soil should have adequate strength, be relatively incompressible so that future settlement is not significant, be stable against volume change as water content or other factors vary, be durable and safe against deterioration, and possess proper permeability.[3]
To ensure adequate soil compaction is achieved, project specifications will indicate the required soil density or degree of compaction that must be achieved. These specifications are generally recommended by a geotechnical engineer in a geotechnical engineering report.
Determination of adequate compaction is done by determining the in-situ density of the soil and comparing it to the maximum density determined by a laboratory test. The most commonly used laboratory test is called the Proctor compaction test and there are two different methods in obtaining the maximum density. They are the standard Proctor and modified Proctor tests; the modified Proctor is more commonly used. For small dams, the standard Proctor may still be the reference.[5]
There are several means of achieving compaction of a material. Some are more appropriate for soil compaction than others, while some techniques are only suitable for particular soils or soils in particular conditions. Some are more suited to compaction of non-soil materials such as asphalt. Generally, those that can apply significant amounts of shear as well as compressive stress, are most effective.
Soil compactors are used to perform test methods which cover laboratory compaction methods used to determine the relationship between molding water content and dry unit weight of soils. Soil placed as engineering fill is compacted to a dense state to obtain satisfactory engineering properties such as, shear strength, compressibility, or permeability. In addition, foundation soils are often compacted to improve their engineering properties. Laboratory compaction tests provide the basis for determining the percent compaction and molding water content needed to achieve the required engineering properties, and for controlling construction to assure that the required compaction and water contents are achieved. Test methods such as EN 13286-2, EN 13286-47, ASTM D698, ASTM D1557, AASHTO T99, AASHTO T180, AASHTO T193, BS 1377:4 provide soil compaction testing procedures.[7]
Soil compaction changes pore space size, distribution and soil strength. One way to quantify the change is by measuring the bulk density. As the pore space decreases within a soil, the bulk density increases. Soils with a higher percentage of clay and silt, which naturally have more pore space, have a lower bulk density than sandier soils.
Bobcat is dedicated to breaking down your barriers to productivity and profit with a tough lineup of light compaction equipment. Our reversible plate compactors (three models), forward plate compactors (three models), rammers (two models) and the mighty TR75 trench roller are powerful, robust, easy to operate and simple to maintain. The design advantages built into every Bobcat light compaction machine help operators accomplish more with less effort.
Compaction algorithms constrain the LSM tree shape. They determine which sorted runs can be merged by it and which sorted runs need to be accessed for a read operation. You can read more on RocksDB Compactions here:Multi-threaded compactions
Here we present a taxonomy of compaction algorithms: Classic Leveled, Tiered, Tiered+Leveled, Leveled-N, FIFO. Out of them, Rocksdb implements Tiered+Leveled (termed Level Compaction in the code), Tiered (termed Universal in the code), and FIFO.
While write amplification is usually worse with leveled than with tiered, there are a few cases where leveled is competitive. The first is key-order inserts and a RocksDB optimization greatly reduces write-amp in that case. The second one is skewed writes where only a small fraction of the keys are likely to be updated. With the right value for compaction priority in RocksDB compaction should stop at the smallest level that is large enough to capture the write working set -- it won't go all the way to the max level. When leveled compaction is some-to-some then compaction is only done for the slices of the LSM tree that overlap the written keys, which can generate less write amplification than all-to-all compaction.
Leveled-N compaction is like leveled compaction but with less write and more read amplification. It allows more than one sorted run per level. Compaction merges all sorted runs from Ln-1 into one sorted run from Ln, which is leveled. And then "-N" is added to the name to indicate there can be n sorted runs per level. The Dostoevsky paper defined a compaction algorithm named Fluid LSM in which the max level has 1 sorted run but the non-max levels can have more than 1 sorted run. Leveled compaction is done into the max level.
caa09b180b