Tuesday, March 18, 2014

"thread.error: can’t start new thread" workaround, CentOS yum, /etc/hosts.deny , echo \n and printf >> append file

BACKGROUND:

While observing network activity on a VPS/proxy server of mine, using the command

netstat -vT

to see a view of current connections and checking on recent failed or succeeded attempts to log in or log on to the system using commands like 

cat /var/log/secure | grep 'Accepted password for root' | tail

and

cat /var/log/secure | grep 'Failed password ' | tail -n 20

and such,

I noticed what appeared to be someone running an automated script or macro or bot or program which was trying to gain access to the system by repeated logon attempts using what I assume someone thinks are the most commonly used usernames and passwords.  I suppose this would be called hammering, or perhaps a remote brute-force password discovery attack of some kind.

Even with an automatic method like this, the chances of someone gaining access in this way were slim to none in this situation.  My password is surely not on the list of "most commonly used passwords ever in the history of the Earth," unless it's an extremely long list that somehow includes the majority of passwords used in contemporary terrestrial history.

The offending party was only racking up about 1 attempt every 4 seconds, so yeah, there was not much chance at all of their hitting it blindly or randomly like that.

However nonthreatening it may have been, this way, it was still bothering me even if only because of how my logs were being filled up with garbage and becoming harder to read and monitor.

I got the idea that it should be a simple task to block the person's domain or IP address and preserve the simplicity of my security logs.  I figured that this login service was probably operating with the TCP protocol or at least within a TCP wrapper, so I just navigated to the /etc folder and examined the "hosts.deny" file first.

I'm terrible with using "vi" to edit so I tried to edit it using "nano" instead.  Apparently "nano" was not installed, I got one of those "command not found" errors.  D'oh!

(I actually added the entries I'd wanted to into the "hosts.deny" file without using vi and without using nano or any other text editor, at this point.  I'll describe that below but first I'll recount the tale of figuring out how to get nano in and working.)

Well this was a perturbing obstacle at first because I'd never actually bothered to check what operating system I was operating in with that system, so I had to determine which distribution of Linux I was using before I could use a package manager to install nano via the command line.  While in the /etc directory, I used the command

ls | grep 'release'

which returned the line "redhat-release" , and upon using the "cat" command to see what tasty little tidbits of information were waiting inside that little file, it returned with

"CentOS release 5.9 (Final)"

which was very helpful in determining that the package management utility I'd want to use is called "yum," which is cute enough.   ...Maybe too cute, that's an even cuter name for a command than "pacman" isn't it?

FOREGROUND:

I tried to install the package for "nano" using the command 

yum install nano

but instead of the computer telling me how happy it was to have done what I'd wanted, it gave me some backtalk error like this:

"thread.error: can’t start new thread"

So my first thought was that I must be severely limited in the number of processes or threads I could run with my user account, which was a disappointing thought because I was logged in as the one and only "root" himself.  I punched-in a command to have it tell me the maximum number of processes that I am limited to running at a time, 

ps -L -u root | wc -l

which returned with "15," and that gave me pause because even though it seemed like a low number for that, I still couldn't really believe that "yum" would require more than one or two simultaneous processes just to install nano, even if it were simply a forking front-end for "rpm" or something.

[2015 edit: I believe the above command returns the number of currently-running processes rather than the result of actually probing for limitation]

I quickly found another page which explained that this error was [at least in this kind of situation] usually due to  a lack of free memory, or available RAM, and not due to a limitation on the number of separate threads or processes that a user is allowed to be running at once.

The solution, I read, was to disable a plugin for yum called "fastestmirror" which is supposed to determine which mirror or which source of package files is the fastest one.  There were several ways listed for disabling the plugin, and I chose to try the method which is to simply use an option in the command line, like so:

yum --disableplugin=fastestmirror update

After issuing this command, I found it was trying to update the entire OS or the entire collection of installed packages, which was not what I'd wanted.  I decided to just let it finish, but it procured another error message and failed anyhow.

I tried again with the appropriate operation for installing nano, instead of the "update" operation, thusly:

yum --disableplugin=fastestmirror install nano

Sadly, that also failed to perform as I'd wished, but without terminating.  It just stopped and never completed the command.  After waiting long enough for me to determine that it was not going to finish, I simply logged in to the system remotely again with another terminal window and issued a "reboot" command.  That worked splendidly for putting an end to the "yum" command, and I entertained hopes that it would clean up any mess left behind by yum's couple of difficulties.

After waiting for long enough to satisfy myself that it'd had enough time to reboot, I logged in again and issued the same command as shown above for disabling the "fastestmirror" plugin and installing "nano."

This time, it worked!  I ran the nano program, one of my favorites, and was very pleased.  I would have then continued to edit the "hosts.deny" file if not for the fact that I'd already added the badguys' IP to that file and successfully blocked the would-be intruder.

FLASHBACK:

To add entries to the "hosts.deny" file without a text editor I figured I'd just use a console command to print the appropriate line of text to the console but then redirect that console output to the end of the target file.

The line I wanted to add to "/etc/hosts.deny" [with the "\n" symbolizing an EOL/newline and with a funnier-sounding domain name than I was really blocking that day] is the following:

ALL: *.koreanbunghole.net\n

The "echo" command didn't seem to want to process the "\n" as a newline, when I was testing it without the redirect-append operator, so instead I used the "printf" command as follows (note this was all being done from within the "/etc" directory):

printf "ALL: *.koreanbunghole.net\n" >> hosts.deny

In order to refresh my memory and familiarize myself again with all the functions etc. being employed here, I first tried it on a test file called "testfile.txt.txt" instead of on the actual "hosts.deny" file a few times, and then used "cat" to see what was in the test file and confirm my expectations.  Having to work without a text editor breeds caution, I guess.

To be honest: initially I'd neglected to include the asterisk and first period, and I found when I looked at the /var/log/secure file that they were still trying.   I appended a second line using their actual IP address instead of the domain name, checked again, and sure enough I found that they had been stopped by that.



Tags: Failed password , thread.error: can’t start new thread , Accepted password for root , /var/log/secure , login attempts , yum , rpm , vi , nano , CentOS , redhat , release , Linux , VPS , ssh, tunneling , proxy , echo , printf , \n , newline , new line , EOL 

No comments:

Post a Comment