47

I have read every thread I could find -- even the ones that say that the question has been answered elsewhere but none of them address the specific issue I am having. Update Manager ran and produced errors relating to unmet dependency and suggested using apt-get install -f which fails with this message:

Unpacking linux-headers-3.5.0-36 (from .../linux-headers-3.5.0-36_3.5.0-36.57~precise1_all.deb) ...
dpkg: error processing /var/cache/apt/archives/linux-headers-3.5.0-36_3.5.0-36.57~precise1_all.deb (--unpack):
 unable to create `/usr/src/linux-headers-3.5.0-36/arch/arm/mach-iop32x/include/mach/glantank.h.dpkg-new' (while processing `./usr/src/linux-headers-3.5.0-36/arch/arm/mach-iop32x/include/mach/glantank.h'): No space left on device

I suspect that the last 5 words No space left on device are significant but df and du both show adequate space. So I am wondering, what device needs more space?

Ouput from df -h

Filesystem 
Size Used Avail Use% 
Mounted on /dev/sda1 5.5G 4.4G 786M 86% 
/ udev 996M 4.0K 996M 1% 
/dev tmpfs 402M 880K 401M 1% 
/run none 5.0M 0 5.0M 0% /run/lock none 1004M 156K 1004M 1% 
/run/shm /dev/sdb1 30G 1.4G 27G 5% 
/home /dev/sdc1 299G 31G 268G 11% /media/HD-PCTU2
Mitch
  • 106,657
  • 24
  • 210
  • 268
Don A
  • 571
  • 1
  • 4
  • 4
  • Do you have space on the root partition? – don.joey Jul 08 '13 at 13:38
  • 1
    Please paste (i) the output of `df -h`and (ii) the output of `mount`. dpkg will clean up the files after it finished working, so it might appear that you have enough space when in fact you don't. Consider running `apt-get clean` before you start dpkg. – January Jul 08 '13 at 13:41
  • 1
    to monitor do `watch -n 1 df -h` in a terminal while doing the `apt-get install` (it will slow down a bit you i/o but you will know which file system is too small) – Emmanuel Jul 08 '13 at 13:51
  • Problem occurring also with Ubuntu 14.04 and kernel version 3.13.0.108 in one computer out of three. I started off with http://askubuntu.com/questions/223143/broken-package-after-update-linux-headers-error-brokencount-0 but the answers shown here are in the same line. – XavierStuvw Feb 05 '17 at 16:58
  • https://askubuntu.com/questions/911865/no-more-disk-space-how-can-i-find-what-is-taking-up-the-space – Deepak Oct 06 '20 at 07:29

5 Answers5

52

I just encountered this same problem. I elsewhere came across a mention of inodes, and ran in my terminal to check inode usage : 

  df -i

This showed inode use at 99%. So, while my disk had plenty of space left, I wasn't able to create more files because of the limit in the number of inodes. Some disk cleanup was the solution to my problem.

Wilf
  • 29,694
  • 16
  • 106
  • 164
Kevin
  • 621
  • 5
  • 3
  • 8
    I had the same problem caused by kernel updates which do not remove old kernels automatically. Linux headers of every kernel update occupy a considerable number of inodes! --- My "safe" solution was to move two kernel headers directories to a different filesystem: `cd /usr/src ; cp -a linux-headers-3.2.0-3{2,3}* /home/tmp-hdr ; rm -r linux-headers-3.2.0-3{2,3}*`, `apt-get-install -f`, remove some other headers the right way: `apt-get purge linux-headers-...`, move the headers from the tmp back: `cd /usr/src ; cp -a /home/tmp-hdr/* .`. Remove more of the old kernels using `apt-get purge`. – pabouk - Ukraine stay strong Jun 29 '14 at 20:40
  • 1
    Please consider participating on the bug reports with low activity: Bug #1089195 [linux-headers will eat your inodes on LTS.](https://bugs.launchpad.net/ubuntu/+source/update-manager/+bug/1089195), Bug #690911 [Installation without formatting fails to remove old kernels](https://bugs.launchpad.net/ubuntu/+source/ubiquity/+bug/690911) – pabouk - Ukraine stay strong Jun 29 '14 at 20:43
  • @pabouk I wonder, if you could have used `dpkg --purge linux-headers-3.2.0-3{2,3}*; apt-get -f install` so that no copying was needed? – jarno Aug 18 '16 at 08:18
  • @jarno It was a long time about but I think that all the commands using the dpkg or apt database were failing. I think it was because of broken dependencies and other inconsistencies caused by insufficient number of inodes during an automatic upgrade. – pabouk - Ukraine stay strong Aug 18 '16 at 13:21
  • The required disk cleanup could be done by `sudo linux-purge --fix` once the [script](https://www.bountysource.com/issues/38300038-feature-request-the-command-should-work-like-this) is published. – jarno Mar 25 '17 at 14:02
  • How do you remove packages since `dpkg` won't work because of no space left? I think only pabouk's solution (manually `mv` `cp` etc) is possible – chefarov May 11 '18 at 11:00
  • pabouk's solution does work: `dpkg --purge ` does remove straight away, even if there is no space left on the device. – Adrian Zaugg Mar 22 '21 at 22:48
47

I'm thinking your boot partition is full of old kernel images, leaving no room for the new one apt-get is trying to install. You can type 

dpkg -l linux-headers-\* linux-image-\* | grep ^ii

into a terminal window. When I do this, I get

ii  linux-headers-3.8.0-18                    3.8.0-18.28                            all          Header files related to Linux kernel version 3.8.0
ii  linux-headers-3.8.0-18-generic            3.8.0-18.28                            amd64        Linux kernel headers for version 3.8.0 on 64 bit x86 SMP
ii  linux-headers-3.8.0-19                    3.8.0-19.30                            all          Header files related to Linux kernel version 3.8.0
ii  linux-headers-3.8.0-19-generic            3.8.0-19.30                            amd64        Linux kernel headers for version 3.8.0 on 64 bit x86 SMP
ii  linux-headers-3.8.0-21                    3.8.0-21.32                            all          Header files related to Linux kernel version 3.8.0
ii  linux-headers-3.8.0-21-generic            3.8.0-21.32                            amd64        Linux kernel headers for version 3.8.0 on 64 bit x86 SMP
ii  linux-headers-3.8.0-22                    3.8.0-22.33                            all          Header files related to Linux kernel version 3.8.0
ii  linux-headers-3.8.0-22-generic            3.8.0-22.33                            amd64        Linux kernel headers for version 3.8.0 on 64 bit x86 SMP
ii  linux-headers-3.8.0-23                    3.8.0-23.34                            all          Header files related to Linux kernel version 3.8.0
ii  linux-headers-3.8.0-23-generic            3.8.0-23.34                            amd64        Linux kernel headers for version 3.8.0 on 64 bit x86 SMP
ii  linux-headers-3.8.0-25                    3.8.0-25.37                            all          Header files related to Linux kernel version 3.8.0
ii  linux-headers-3.8.0-25-generic            3.8.0-25.37                            amd64        Linux kernel headers for version 3.8.0 on 64 bit x86 SMP
ii  linux-headers-generic                     3.8.0.25.43                            amd64        Generic Linux kernel headers

Since I have quite a few, I could delete some of the oldest with

sudo apt-get remove linux-headers-3.8.0-18

If apt-get fails because it misses some package dependencies (which you can't install due to the lack of free inodes), you can resort to dpkg directly instead:

sudo dpkg --remove linux-headers-3.8.0-18

Do that a couple of times to your oldest linux-headers-* and you should have room.

David Foerster
  • 35,754
  • 55
  • 92
  • 145
Marc
  • 9,870
  • 2
  • 21
  • 28
  • 4
    This brings to light the other issue. Every time I attempt to run dpkg it tells me that it has an unmet dependency The following packages have unmet dependencies: linux-headers-3.5.0-36-generic : Depends: linux-headers-3.5.0-36 but it is not going to be installed and that is the thing that won't install because there isn't enough space – Don A Jul 08 '13 at 14:41
  • Perhaps a backup of your user files is in order, before you mess with your boot partition? I've never had trouble doing this, but if I did, I'd have everything I needed for a reinstall. Of course, I do a backup every day anyway. – Marc Jul 08 '13 at 14:55
  • I was hoping I wouldn't need to mess with the boot partition and that there was a way to break out of the endless cycle of dependency. I see now that it's probably going to be easier to backup data and start over. Now that I know the importance of cleaning out older items I'll make it a habit. Thanks to all for their assistance. – Don A Jul 08 '13 at 16:44
  • Not at all . . . backing up is always a good idea, and you're not really messing with the boot partition very much, just uninstalling some packages with apt-get. It should be a piece of cake. I just suggested backing up your data because I always suggest backing up data if you're having a problem (or even if you're not!) I actually ran those commands on my own system to test them. It took me three minutes to remove `linux-headers-3.8.0-18`. – Marc Jul 08 '13 at 16:54
  • 1
    If it fails again for `sudo dpkg --remove linux-headers-3.8.0-18`, remove `sudo dpkg --remove linux-headers-3.8.0-18-generic` and it will take the other header with it. – Sam Dec 12 '17 at 14:43
  • you can't use `dpkg` if no space is left – chefarov May 11 '18 at 10:59
21

I used sudo apt-get autoremove and it removed a bunch of old kernel-headers packages. Good to go after that.

Eliah Kagan
  • 116,445
  • 54
  • 318
  • 493
Frank Sfalanga Jr.
  • 211
  • 2
  • 2
  • In my case apt-get refused to do autoremove giving precedence to the fact that there was a broken package to sort out. Which broken package had been caused by the inode saturation. See also [this AskUbuntu thread](http://askubuntu.com/questions/223143/) – XavierStuvw Feb 05 '17 at 16:57
  • Well, this just saved me! After an update my x session was never entering, just asking to log in again and I oft got the no space left message. I thought I had to format my hd, but your trick just saved me :) – user5193682 May 17 '17 at 21:44
  • 1
    Lol I can't run this because I have too many files... – Brian Leishman Dec 14 '18 at 17:21
13

As far as I know the limit of the inodes in Linux ext* file systems is around 4 billion, but not less, which is huge amount of files. So your problem is that something is generating a lot of files. I can propose you to check first this: http://blog.scoutapp.com/articles/2014/10/08/understanding-disk-inodes So you will find what is filling the inodes of your FS. I can propose also to clean the apt cache:

sudo apt-get clean

and

sudo apt-get autoclean
Stefan
  • 389
  • 2
  • 6
0

If you ran out of disk space, apt-get clean / apt-get autoclean doesn't give you enough space, and you have some RAM to spare, try my answer at:

https://unix.stackexchange.com/a/726355/250404

I used a ramdisk to squeeze out enough space to complete apt --fix-broken install and recover from this.

karel
  • 110,292
  • 102
  • 269
  • 299
Cedar
  • 101
  • 2