Windows OS Hub / Windows 8 / Activation of Data Deduplication in Windows 8.1

January 29, 2014 Windows 8

Activation of Data Deduplication in Windows 8.1

It is to be recalled that activation of a built-in functionality of data deduplication on disk partitions appeared in Windows Server 2012. The aim of deduplication technology is disk space saving on file servers by means of searching and deletion of duplicated data. I.E. the system is searching the same blocks on NTFS file system volume and deleting them, replacing such blocks with the links to the original source, clearing disk space effectively. This technology is focused on server market segment and is not available for client version of Windows 8 even in Pro and Enterprise editions.

However, there is a trick which allows to activate deduplication technology in Windows 8 (and Windows 8.1).  Some enthusiasts extracted corresponding packages from Windows Server 2012 / R2 to make it possible.

Note. Officially, the deduplication functionality is not supported by client OS and for this reason it is considered to be unsupported configuration. However by activating deduplication functionality for such tasks as Hyper-V virtualization testing environment or for disk which serves  for images and different software installation packages storage you can save up to 90% of disk space which is impossible to achieve with the usage of NTFS compression or Single Instance Storage( which has become the thing of the past already).

  1.   Download and un-zip the archive with a file which contains a rollup for activation of deduplication
  2. Perform the following commands (with the administrative privilege)
    • For Windows 8: 
      1
2

      dism /online /add-package /packagepath:Microsoft-Windows-VdsInterop-Package~31bf3856ad364e35~amd64~~6.2.9200.16384.cab /packagepath:Microsoft-Windows-VdsInterop-Package~31bf3856ad364e35~amd64~en-US~6.2.9200.16384.cab /packagepath:Microsoft-Windows-FileServer-Package~31bf3856ad364e35~amd64~~6.2.9200.16384.cab /packagepath:Microsoft-Windows-FileServer-Package~31bf3856ad364e35~amd64~en-US~6.2.9200.16384.cab /packagepath:Microsoft-Windows-Dedup-Package~31bf3856ad364e35~amd64~~6.2.9200.16384.cab /packagepath:Microsoft-Windows-Dedup-Package~31bf3856ad364e35~amd64~en-US~6.2.9200.16384.cab
dism /online /enable-feature /featurename:Dedup-Core /all
    • For Windows 8.1: 
      1
2

      dism /online /add-package /packagepath:Microsoft-Windows-VdsInterop-Package~31bf3856ad364e35~amd64~~6.3.9600.16384.cab /packagepath:Microsoft-Windows-VdsInterop-Package~31bf3856ad364e35~amd64~en-US~6.3.9600.16384.cab /packagepath:Microsoft-Windows-FileServer-Package~31bf3856ad364e35~amd64~~6.3.9600.16384.cab /packagepath:Microsoft-Windows-FileServer-Package~31bf3856ad364e35~amd64~en-US~6.3.9600.16384.cab /packagepath:Microsoft-Windows-Dedup-Package~31bf3856ad364e35~amd64~~6.3.9600.16384.cab /packagepath:Microsoft-Windows-Dedup-Package~31bf3856ad364e35~amd64~en-US~6.3.9600.16384.cab
dism /online /enable-feature /featurename:Dedup-Core /all

      install deduplication feauture in windows 8.1

    Note. As deduplication packages are extracted from Windows Server 2012/R2  they will be working only on 64 bit versions of Windows 8.
  3. After feature installation the new, previously non-existent option File Server Role -> Data Deduplication will appear in “Windows Features”( Control Panel\All Control Panel Items\Programs and Features\Turn Windows Features on or off section)

windows 8 features: data deduplication

Deduplication management  in Windows is performed via PowerShell, launched with administrative privileges (of course graphical Management utility which is included in Server Manager console won`t operate)

To enable deduplication for volume D:, perform the following command: 

1

Enable-DedupVolume -Volume D:
Tip. It is impossible to enable deduplication for system or boot volume.

It is possible to exclude some catalogues from duplicated data on disk in Windows 2012 R2 (PowerShell 4.0):

1

Set-DedupVolume -volume D: -ExcludeFolder d:\HomeFolder;d:\mp3

To enable deduplication process immediately, perform the following command:

1

Start-DedupJob –Volume D: –Type Optimization

To get information about the status of the process use the following:

1

Get-DedupStatus

powershell Get-DedupStatus

Note. The speed of deduplication performance on average hardware is about 90 Gb/hour as stated in official sources.

In case of deduplication feature installation in Windows 8, three new tasks are created in a Task Sheduleler, they perform operation of data deduplication on volumes according to the definite schedule. Task parameters can be viewed or changed via graphical Sheduler interface. Sheduler -> Task Sheduler Library -> Microsoft -> Window -> Deduplication).

deduplication task in task sheduller library

or via PoSh:

1

Get-DedupSchedule

powershell cmdlet: Get-DedupSchedule

Tip One should be aware that deduplication partly reduces system performance. It`s normal because system has to use  additional system resources for the search of the same blocks on volume. That`s why it`s better to create such a schedule according to which the deduplication  is performed during hours which are considered to be nonworking for PC user.

For that files in which there were no changes for more than 5 days the process of deduplication is enabled by default. In most cases (particularly in test environment) it`s better to enable deduplication for all files regardless to the time of their changing. 

1

Set-Dedupvolume D: -MinimumFileAgeDays 0

After the deduplication finishing all its results (which involve saving of disk space) can be estimated with a help of PoSh:

1

Get-DedupStatus

Or visually in disk properties

Important. When you enable data deduplication for some folders, they can not be read on another PC if it is not installed deduplication component (even though you will see a list of files on the disk, but they can not be read).

If the other computer can not install the deduplication, you must disable data deduplication on a volume:

1

Disable-DedupVolume-Volume D:

4 comments
0
Facebook Twitter Google + Pinterest

Related Reading

How to Restore Deleted EFI System Partition in...

July 29, 2020

Internal SSD/SATA Drive Shows as a Removable in...

December 16, 2019

0x80092004: .NET Framework Install Error on Windows Server

September 16, 2019

Fix: The Local Print Spooler Service Not Running...

July 9, 2019

Resetting Windows Update Agent Settings

May 31, 2019

4 comments

Francisco Ricardo August 22, 2014 - 8:23 pm

I use Win 8.1 Build 6.3.9600.17031, where I can locate cabs these version to download???

Reply
Max August 27, 2014 - 6:56 am

Try package for Windows 8.1 – windows81-dedup.zip

Reply
wes April 30, 2015 - 5:15 pm

I’m in the same boat. I get an error when trying to use 16384 cabs. I need 17031 cabs. Anybody got a clue how to extract new cabs for the new dism modules

Reply
Dennis Prins July 13, 2015 - 11:43 am

When your DISM asks nummers for  6.3.9600.17031, and say “This package is not acclicable to this image”
You first remove them en then try again.
Her is the remove command.. Run it as Admin from anywere

dism /online /Remove-Package /packagename:”Microsoft-Windows-VdsInterop-Package~31bf3856ad364e35~amd64~~6.3.9600.16384″ /packagename:”Microsoft-Windows-VdsInterop-Package~31bf3856ad364e35~amd64~en-US~6.3.9600.16384″ /packagename:”Microsoft-Windows-FileServer-Package~31bf3856ad364e35~amd64~~6.3.9600.16384″ /packagename:”Microsoft-Windows-FileServer-Package~31bf3856ad364e35~amd64~en-US~6.3.9600.16384″ /packagename:”Microsoft-Windows-Dedup-Package~31bf3856ad364e35~amd64~~6.3.9600.16384″ /packagename:”Microsoft-Windows-Dedup-Package~31bf3856ad364e35~amd64~en-US~6.3.9600.16384″

Reply

Leave a Comment