https://www.lifewire.com/free-hard-drive-testing-programs-2626183
Most companies like WD will have one of their own, you can do a light surface scan, a good deep scan can take hours.
Depends also how old the drive is, if it's brand new, less chance, if it's already a couple of years..
You said you already defragged, so that can't be it.
Comment has been collapsed.
Thanks I will do the Seagate test but it says to backup first as if there is a problem there is no way back and you have to replace the disk directly (if I understand correctly ) Well that is a bit aggressive as I just want to try my new game atm but I will do it when I have some more time and a very good backup.
Comment has been collapsed.
Can't say about Seagate (i got WD) but normally most would just scan and not try to change or fix anything and should be safe.
Comment has been collapsed.
If it is a HDD, a defrag should be in order.
But yeah, if a game has a zillion small files, allocating them can take a looooooong time.
Comment has been collapsed.
Never happened to me before
Found these threads reporting same issue https://forums.anandtech.com/threads/steam-allocating-disk-space-taking-much-longer-than-it-used-to.2420952
https://linustechtips.com/main/topic/653541-why-does-steam-allocating-disk-space-takes-ages
Comment has been collapsed.
Thanks no solutions but at least I am not the only one.
Comment has been collapsed.
I wouldn't know. It happens to me when downloading some games, while other games are fine. I have a regular generic HDD, so I don't think it's really a drive issue. I'm sure that a SSD would do much better, but I don't think it means anything is "broken".
Comment has been collapsed.
Try to defragment the drive. Here are some free tools
Comment has been collapsed.
With fragmentation ruled out, I'd check for other disk activity - disk backups or antivirus scans going on at the same time, for example, will distract your computer from allocating space for the game.
If you have multiple disks, you can try creating a steam library on another disk and seeing if it performs any better. If you're dealing with limited space, such as on an SD, you can download to one Steam library more quickly, play it there, then copy the game to your HDD later to free up space. You can delete the game's local content and then click Install, pointing it to the library on the HDD. Steam will find the existing game files (including saves, DLC, bonus content, etc.) and use that instead of downloading the package again.
Comment has been collapsed.
Back up is only when not in use and my antivirus was not scanning. I only have 1 hybrid disk so I can not try the other option but thanks for your suggestions.
The strange part is I never had this before only the last few weeks. I thought it might had something to do with 4 or 5 games ready for download at the same time. But tonight it was only 1 game that took 36 min to allocate! Crazy.
Comment has been collapsed.
Hybrid drives can behave a little differently from conventional caching drives, so that's good to know. Further advice if the problem returns / continues:
- If you are on a notebook, check the power plan. If you're not plugged in and on anything less than Performance, that can be a factor in disk performance.
- Try clearing your Steam download cache. It's under Settings > Downloads. It sounds like that would affect downloads more than disk space allocation, but I don't know how Steam's implemented it. Disk space allocation could be tied to content that's already in the cache, especially if this is a restarted download, and clearing it can give it a fresh start.
- SSD/HDD management on a hybrid drive is performed by firmware. You can check Seagate's release notes for a firmware update or a known-problems list. I would recommend considering an external backup drive before updating the firmware, if you don't have one already.
- Obligatory: Have you tried turning it off and on?
Comment has been collapsed.
Thanks for all the tips. I cleared the cache as I never did that before.
Comment has been collapsed.
Either its stuck - shut down the process and retry, or your drive is failing due to wear and tear
or other malfunction. Being extremely fragmented, might also be possible but i doubt that with 550GB free space.
^ If its also your main drive (OS on it) ... it could be another process - viewing videos, downloading something else,
or windows indexing, fucking with its read/write rate at a inconvenient time, slowing down both processes.
You could install some HDD tool (crystal disk info / crystal disk mark) see and test it.
If your drives are from 2007 something or older yup its been 10 years > might be "dying".
Comment has been collapsed.
I will try crystal disk later today. Now it is making a backup. Windows disk check says there is nothing wrong with it.
I have a normal pc that is permanently connected to my TV. This pc is only used for gaming. Sometimes I play a few days in one week and sometimes I don't play for weeks. So on average this pc is used for 2 to 3 hours a week wich is not excessive imo. Ok the disk is 2,5 years old by now but with this usage I expect it to live a bit longer.
There is nothing else on this pc expect eset anti virus. Never had any delays with this very nice and smart programme. I also use it on my business pc which is used for all kinds of things and never any delays. So performing something else is impossible as there is nothing else on there. Thanks for the tips.
Comment has been collapsed.
It's actually an issue with steam. They supposedly fixed that a few months ago but it's still happening.
Comment has been collapsed.
Do you know more about this as this sounds perfectly acceptable. This problem only started a few weeks ago, before that it took several seconds.
Comment has been collapsed.
windows 10 doesn't need defrag software. it's doing that in the background on it's own.
use crystal disk info to check the smart status of the drive(s).
last time i had a slow disk was my crappy seagate barracuda on it's way out. good thing that was just the steam disk and not the one with the valuable stuff. 2TB down the shitter.
Comment has been collapsed.
I will try Crystal disk later today, it is backing up now.
Comment has been collapsed.
FWIW preallocating took me around 7 minutes yesterday when installing DOOM, on an SSD. I've never noticed a preallocating time this long before, it's usually taking only a few seconds. I've done some research into the "problem" but I haven't found a believable explanation yet, just self-proclaimed computer experts rambling on about slow drives and not enough free space etc...
Comment has been collapsed.
first time i seen this thread... i somewhat fixed it by launching the steam client with the "-tcp" commandline parameter. it still can be slow at times, but it is usually still faster then the normal udp connection.
Comment has been collapsed.
That's both interesting and weird since UDP is just the connectionless variant of the transport protocol in the IP-stack and shouldn't have any effect unless you're actually transferring stuff :P
UDP is basically used for stuff where it's not as important if you happen to lose some packets on the way, like in streaming and such, while TCP connections use handshakes and used for important stuff like file transfers. Maybe Steam handles handshaking on the application layer though.
Might it just have been a case of confirmation bias? What made you try this switch in the first place?
Comment has been collapsed.
2016 winter sale, not being able to logon. i just noticed after the swap that i never have seen the allocating issues since. =)
isn't it transferring metadata? :P (to verify ownership)
Comment has been collapsed.
It's definitely something to try at least. I'll see if my somewhat limited google fu will be able to turn something up later
Comment has been collapsed.
Looks like Steam is actually physically writing data to the disk before starting my downloads of at least DOOM, The New Order and The Old Blood. Would you be willing to reinstall a decently large game for research purposes at some point? It would be interesting to hear if the preallocating time seems consistent with the amount of data and your drives write speed :)
Comment has been collapsed.
yeah, that's what a disk allocation is. it's to fill the hdd/sdd with "this data is used for xyz" so that 2/3's the way thru the download you can't run out of space, it already set aside the space it needed. but, they very very first thing steam does when you tell it to install, is check online to insure that the logged in user still/actually owns the game before it installs. if steam is under ddos, or just plain finicky, or on a crap internet, or slammed with an extremely large amount of traffic. using a tcp connection will help it get thru the B.S. to tell valve you actually own it.
if you backup a game with steam backup, uninstall it, then disable the internet while on the steam client. then tell it to restore, it will fail. it needs the internet to verify ownership. same reason offline mode will not allow restoring.
but as far as testing sometime sure. i'd prefer it not be to extremely large though, as i already am a little tight on space. but i'm sure we could find a game i already have installed and just keep deleting/restoring it for testing purposes sometime. =)
edit: had to add sdd, for some reason i stated old school at first only. ^^
edit2: had it backwards.. i said using udp would help, but meant tcp helps. xD
Comment has been collapsed.
No, the purpose of allocating space is just to reserve it in the file allocation table, not to actually write data to the disk. Steam is physically writing data to the disk for some unknown reason and this is what's taking up all the seemingly unnecessary amounts of time we have been noticing.
I think you mean TCP will help because that's a more robust protocol, but this is all beside the point as it doesn't affect the preallocation time.
If you don't mind, just pick a reasonably large game to delete and reinstall. This should be enough to tell if the preallocation takes minutes or seconds. It's not important though, I was just curious about your results since you claimed to have, at least alleviated, the problem. :)
Comment has been collapsed.
the only way possible to reserve space it to fill it with data. and yes, if the allocation process has truly started (after the initial check to insure ownership) and it is still slow, then it has absolutely nothing to do with tcp/udp and it's simply cause it is a large game. i claimed it was a "partial/somewhat" fix because steam does not have the message prior to "allocating disk space" that states "checking online for ownership" that it actually does just before the allocation. if steam servers are finicky like i said, then the tcp connection will help that first step go faster, not the actual allocation process.
basically what i mean is if it is stuck at the very beginning of the allocation with no progression what so ever, then it is likely actually stuck on verification rather then allocation. (steam just doesn't have this message)
yes though, i did mean tcp. that was another early a.m. typo i made. =)
sure thing though, i'll still test around with it a little later today. i think it's time to replay doom soon anyways, so i'll shift some data around and make room for it and install it a few times. ^^
edit: actually to be more accurate, the size of the game doesn't matter.. its the amount of small files that will slow it down more then the sheer size of the game. (1x 1gb file vs 1,000,000x 1kb files)
Comment has been collapsed.
the only way possible to reserve space it to fill it with data.No, you just have to mark it as used in the file allocation table
basically what i mean is if it is stuck at the very beginning of the allocation with no progression what so ever, then it is likely actually stuck on verification rather then allocation.This has not been the case when I've checked on my computer and I'd say that it's highly unlikely that it would ever happen, and not quickly getting fixed, because this is simply not how one would design a process like this.
Just one install would do the trick so it sounds like perfect timing then :)
Comment has been collapsed.
No, you just have to mark it as used in the file allocation table
which in itself is "writing data" telling it that it's to be used by said game.
This has not been the case when I've checked on my computer and I'd say that it's highly unlikely that it would ever happen, and not quickly getting fixed, because this is simply not how one would design a process like this.
i guarantee around winter sale times it happens all the time. there is nothing steam can do to fix millions of people slamming them at once, and this is valve we are talking about on a design process....
Comment has been collapsed.
Technically yes, but the difference is that you write data that amounts to bytes to the file allocation table and not gigabytes of data to the sectors set up for storing user data.
So you're saying that there are several minutes, or even half an hour, delays happening with no timeouts and no need to reinitiate the communication? Seems unlikely but I'll just have to take your word for it. If this is the case they're practically ddosing themselves, or at least not helping since the clients must be trying to continuously make contact with the server during this time.
Comment has been collapsed.
i'm not sure if you saw an edit/addon on one of my prior comments, but yes. what i think is actually going on for you more than anything to do with tcp/udp/internet is the games that are having the slower time, is actually a game with a TON of tiny files. games with tons and tons of small files can be far worse on allocation then games with a few dozen really large files. it allocates each file individually, rather than just taking a chuck of space and setting it aside.
yes. it's a terrible design that has needed an update for the past 5years at least. but tbh, it's not usually a problem except during extremely high load times. during those times the tcp connection can help. that's all i really meant. it won't actually help the allocation at all, beyond getting the connection to valve a little stronger with 2-way handshaking so that it can discover what space needs to be allocated by what files as well as the ownership verification part.
Comment has been collapsed.
This is easy to check. Doom has 428 files in the installation folder which I would be hard pressed to call a ton of files. I know what's happening on my computer now. I'm basically only interested in if this preallocation behavior is somewhat random or if it's the same for everyone, all the time, and we just don't usually notice. I could have sworn that I've never seen more than a few seconds of preallocating time earlier but since there have been people complaining about it for years I don't really know what to make of it.
Comment has been collapsed.
i don't consider 428 many personally. i meant more along the lines of payday 2's '6,995 files'
well the complaints over years have 3 factors really.
-
current load on valve servers. (are they currently slammed with traffic, and replies are wonky)
-
speed of their internet. (how fast valves replies come in and go out)
-
speed of their hdd/sdd. (how fast it can write the allocation)
Comment has been collapsed.
I finally got around to it and during my testing of tcp vs udp, it didn't really seem to make any difference at all.
(following tests were restores of already downloaded content, timer started immediate after "agree" was clicked and stopped as soon as the actual "installing" process began)
[keep in mind these were manual start/stop clicked times, so not 100% accurate]
Game Tested: Just Cause 3
Prepping/Allocating via UDP = 25seconds
Prepping/Allocating via TCP = 22seconds
(Compressed Backup Size = 43.7GB)
Game Tested: Payday 2
Prepping/Allocating via UDP = 24seconds
Prepping/Allocating via TCP = 27seconds
(Compressed Backup Size = 17.4GB)
Game Tested: The Evil Within
Prepping/Allocating via UDP = 1minute 20seconds
Prepping/Allocating via TCP = 1minute 20seconds
(Compressed Backup Size = 34.0GB)
however the tests were not done during a high load time on steam store. if it had, the difference may be more significant. i really dunno anymore. i kinda expected a bit more of a difference as it was.
Laptop: ASUS ROG Strix GL702VM-DB74
Internet: 4.702Mbps DN / 0.892 Mbps UP
edit(s):
cleaner formatting
added compressed backup sizes to show how size by itself doesn't mean anything (it's the amount of files and their compression ratio)
Comment has been collapsed.
It does seem a bit random if one gets long preallocation times, or any preallocation at all even. When installing Space Rangers HD: A War Apart I had no preallocation what so ever but with Homefront: The Revolution there was at least a minute preallocation. I got distracted and lost track of time :P
Edit: I decided to try PayDay 2 as well and I didn't get any preallocation at all, just a second or two of allocating disk space. This is after clicking agree on the license and then the download starts.
Comment has been collapsed.
I've also noticed it lately, even small-ish games installing to SSD still take longer to allocate than large games installing to HDD used to. :)
Comment has been collapsed.
Oh, so it's specifically SSD installations that are troublesome? I never made that connection for some reason
Comment has been collapsed.
Oh, no no no, sorry, what I meant was that lately installing on SSD is even slower than installing on HDD used to be before this problem popped up. But installing on HDD is slower, too. It used to be more or less "instant" in either case.
Comment has been collapsed.
I see, thank you for clarifying. Saved me some time experimenting lol
Comment has been collapsed.
Well, I'm sure that it's not a problem with the drive itself as I'm quite frequently moving large chunks of data between different drives. I'll try the web cache deletion next time I encounter this though. I have 100+ GB or 20+% free space on my system drive where I was installing DOOM and an SSD doesn't fragment like regular drives.
The term "fragmentation" is meaningless in an environment which does not require a physical sequential search of a physical location on a disk. Since there are no moving parts, memory access takes the same amount of time regardless of location. However, the address of each file fragments takes a small amount of space and there are of course limits where this starts to affect performance or even produce errors. This is all taken care of by the OS itself though, at least in my case with Windows 10 :)
I find it weird if it's some kind of communication problem though. Although I don't actually know what constitutes this preallocating, it sounds like it's simply a matter of allocating enough disk space before the download begins. I guess it could be a bit more complicated than that but I have a hard time believing that Steam does some kind of defragmenting to have a continuous block of free space for the download/install.
It's strange that no one seems to be able, or willing, to explain what's actually happening when Steam is preallocating.
Comment has been collapsed.
Because only Valve knows it. It is not like any other space reservation/allocation I have ever seen. Almost as if it reserves space individually for each file it will decompress from the installer, before actually filling the reserved space with the real data.
Comment has been collapsed.
Yes but I wonder why it seems to be treated like some kind of trade secret. Even if it would reserve space for each file individually it wouldn't take this long. In the recent case with the 58.1 GB DOOM install, even writing this amount of data to the drive would take about 2 minutes and I imagine just allocating the space would be a matter of milliseconds, with regard to the drive and its performance.
Comment has been collapsed.
I think you're right... Now that I think about it I also encountered some period when the download server was unreachable. So I wonder if the process goes like:
- Click to install the game
- Steam says "preallocating disk space"
- Steam checks online to see how big the game files are
- Steam allocates the disk space
So if there is some delay in step 3 in contacting the server then it would seem to be allocating for a long time, but really just a network delay in fetching how large the package is.
Comment has been collapsed.
If seven minutes or even up to over half an hour, as in OP's case, is "just a network delay" it's really piss poor programming and I would need very strong evidence if I ever were to believe anything like that :P
Edit: Also, the client already knows the size of the package while preallocating. This try the time went up from 7 to 10 minutes, on a normal drive.
Comment has been collapsed.
At one point I changed my steam download server to a different region because it was just stuck there "forever"
Comment has been collapsed.
Turns out Steam is actually physically writing something to the hard drive after all. Which, to me, is strange but explains the 10-minute duration almost perfectly as this is an older WD Red drive, with some Virtual Box images and some other stuff running as well.
Seems obvious now but I couldn't, and still can't, imagine why it would write all this data before starting the download. This also doesn't explain OP's extreme write times or the time on my SSD. I wonder if it's possible to disable this seemingly redundant "feature"
I also don't get why I've never noticed this before and I've seen people complaining about it since at least two years back.
Comment has been collapsed.
It always seemed to do a sparse preallocation, I wonder if they've changed it to actually write zeroes to disk for the full size of the game first? Or if something changed in a library that they use or something. Or if we're all going mad... :P
Comment has been collapsed.
Maybe it's implemented on a game to game basis. I'll have to try to reinstall The Old Blood and The New Order to check because I don't remember any annoying long preallocation with those. I may have selective memory though, has happened before :P
Edit: Yeah, no it's the same with these two. How very odd. Oh well. I'll ask someone claiming to be rid of this "problem" to reinstall a game, for science!
Comment has been collapsed.
If you have not tried it, try clearing your Steam download cache. In the Steam client, go to Steam > Settings. On the Downloads page of the Settings dialog, click the Clear Download Cache button.
You will need to log in again after this - please let me know if you see any difference following this.
Comment has been collapsed.
I will have to give it a try next time I encounter this weird behaviour. I don't know know if it's a reproducible "error" and it seem's like a waste of bandwidth and resources to try lol. (I'm raised by people born in the 19'th century :P )
Comment has been collapsed.
Well, I stopped being lazy and actually checked what was going on with the computer in the preallocating phase. For some unimaginable reason Steam actually does allocate the space physically on the hard drive. In this case, it was writing with a speed of about 100MB/s during the whole 10-minute phase. This is an older WD Red drive and the write speed corresponds almost perfectly to the actual preallocating time. Mystery solved, sort of anyway.
It would be very interesting to know why this is taking place at all as it seems totally redundant to me.
The steam cache reset did nothing to alleviate this problem, as one can imagine with the above knowledge :)
Comment has been collapsed.
By now I thought it had to be the game as sometimes it works fine and sometimes it takes a few minutes (luckily no half an hours anymore). But this cannot be it either. You are talking about Doom that took a long time on SSD and Mully replies it is extremely fast with Doom on SSD. Fact 8s it never occurred with any game for 2 years and lately is does. Steam must be ghosted or something.
Comment has been collapsed.
I'm not really sure of anything at this moment. Only that I will feel extremely silly if it turns out that this is an old and reasonably well-known feature that got implemented years ago but I haven't noticed until now for some reason and even convinced myself that this has not been the case until now. I know that sounds unlikely but I also know that our brains tend to deceive us more often than we would like to know :P
Anyway, I merely wanted to confirm that I also had noticed this behaviour now. Sorry for, sort of, highjacking your thread
Comment has been collapsed.
You did not in anyway high jacked it as you are experiencing the same thing. That's good to know as it makes it more likely it's not my pc. And I am still very curious to read a solution or at least a plausible explanation why this happens all of the sudden.
Comment has been collapsed.
Yeah, this is clearly not the correct term. Non-native speaker writing without engaging neocortex. Maybe co-opting would have been a more suitable term but I'm glad you don't feel like I'm taking over, or something like that.
Comment has been collapsed.
it's extremely fast with a ssd even for huge games like doom, so maybe it's an issue with hdd in general or just yours.
try what others told you about check the hdd with crystaldiskinfo and defragmenting it (ie: with defraggler).
Comment has been collapsed.
Windows has a built in disk/file system checker (chkdsk). Try that before downloading anything extra. You can run it by right clicking the drive in windows explorer, select properties, then under the Error checking, click the Check button. This will only do a quick file system scan though, for a more thorough job you need to run it from an administrator command prompt.
After opening an administrator command prompt, type:
chkdsk [drive letter] /F /R
This will scan the entire disk for bad sectors and recover data if possible, then mark the sectors unusable by the file system. I would do this first before running the disk manufacturer's diagnostic utility as it will just hide the bad sector in the drive's firmware and you will lose any data on that sector. If you run chkdsk /F /R and it does finds bad sectors, then run your disk manufacturer's diagnostic utility and let it scan for bad sectors to. If the drive is under warranty, it should have a way to save the log from the scan and you can get the drive replaced.
Comment has been collapsed.
Using the Windows disk checker was the first thing I tried and it says there are no problems. The topic is over a month old and without any changes to my pc it works fine most of the time now. Sometimes it still takes a bit longer but mostly a few minutes while that used to be seconds. I getting more and more convinced it must have been a Steam change or something.
Thanks for your suggestion.
Comment has been collapsed.
Oops. I didn't notice the OP date. I've never had that problem with downloading new games, but some games do that when patching. Payday2 is the worst. It will download a small patch, but that small patch may change 25+GB worth of files, so Steam has to make a copy of that 25GB, then apply the patch. The patch only takes a few seconds, but it will grind the HD for 10 minutes while it patches.
Comment has been collapsed.
So does anyone have an idea what's causing this a year later?
My allocating times are not on the level the OP's are but it still takes easily an hour+ for games like Hitman that's 70GB and Total war war titles that tend to be 50GB+.
In my case it isn't the drive either, it's a rather new SSD drive with enough free disk space, my torrent client is set to pre-allocate as well and does this is seconds unlike steam even when downloading large files.
I'm wondering if this is a steam related problem or a win10 settings problem, too bad I deleted my secondary OS (win7) or I could have tested if it was as slow on both operating systems.
Comment has been collapsed.
I have a descent PC and fast internet but lately allocating disk space is taking ages. For example I am trying to download a 4,6 GB game with 550 GB free disk space. By now it is allocating free disk space for over half an hour! This is driving me crazy.
What could be the problem?
Comment has been collapsed.