Often when I want to find a domain or country of origin based on an IP, or find such kinds of information based on a domain name, I'll use a handy little website called DNSgoodies, but it occurs to me that on a typical Linux system there are already commands available to provide such information as it is available from the DNS servers being used (and that's assuming there isn't a well-informed DNS service running on the local system, which makes it even easier or less network-intensive to perform such a query locally.)
The good old command "nslookup" is usually available in Linux as well as in Windows, though it's much more useful for finding the IP address based on a known domain name, or for testing whether or not DNS-lookup functions are working properly, and less useful for finding information about a domain or origin based on a known IP address (at least as far as I know.)
One command for doing it the other way, e.g. for finding an unknown domain name from a known IP address, is simply "host." Host works both ways, it can provide an IP from an inputted name, or it can provide a name from an inputted IP. Typing a command such as the following should return a relevant domain name:
host 98.138.252.30
It returns something with:
ir2.fp.vip.ne1.yahoo.com
I found that IP address with the command:
nslookup www.yahoo.com
Another command that can be used this way is "dig," commonly used with an "-x" switch and sometimes with some switches to shorten the results etc. like "+noall" and "+answer" as well, like so:
dig +noall +answer -x 98.138.252.30
To be honest I'm not entirely clear on all of the subtle differences between all these commands nor about the exact meanings of all the possible information they provide, at this time, but for my purposes it's good enough info to make note of.
Friday, March 28, 2014
Tuesday, March 25, 2014
Cloud Storage for Linux part 1
Cloud storage for Linux users is a little different than it is for users who are on the more "mainstream" operating systems like Windows, Mac OS, or Android, but it's not necessarily more limited.
Today I was researching cloud storage services which cater specifically to Linux, meaning they have an official client program for use on a Linux operating system. Surprisingly, I found there are very few that offer any free service as well as a client or official support for Linux: Dropbox, Wuala, Spideroak, Ubuntu One, Minus, etc.
There are several major services which support Linux directly in this way if you will to pay them, such as JustCloud, SugarSync, etc., though personally I find the free services to be much more interesting and useful for the sake of having multiple/redundant backups without it costing too many times the price of just having an awful lot of physical hard disks hanging around.
Fortunately for users of Linux, having a client program or application specific to each internet-based storage service is probably not as important as it is to users of the aforementioned mainstream OS's.
Linux users can simply choose services which offer access via the more standardized transfer protocols such as FTP, WebDAV, Swift, S3, etc., and then choose the cloud-storage providers that are accessible and usable in these ways. There is really no need for an OS-specific client program when the service offers/allows access through these avenues, because there are already so many FTP clients [for example] that are available for Linux.
That being said, it's no small wonder that there aren't more service-specific clients available for Linux. It can be intimidating indeed, from a software development point of view, to try to support Linux and BSD and other OS's like that because the distributions and flavors of Linux are so varied and often different in ways that can be subtle and/or obtuse. I think, though, that this kind of reason to avoid it is not well founded, especially considering that packages can be distributed or obtained in a "noarch" format that is specific to an OS or distro (i.e. Debian) but independent of any hardware types or architectures, as well as a "portable binary" kind of format that is specific to the hardware architecture but not constrained by any brand or distribution of Linux, as long as it is indeed Linux.
It seems to me that with those two options, it should be a relatively small challenge to make software for Linux that can be used with most any distribution.
Today I was researching cloud storage services which cater specifically to Linux, meaning they have an official client program for use on a Linux operating system. Surprisingly, I found there are very few that offer any free service as well as a client or official support for Linux: Dropbox, Wuala, Spideroak, Ubuntu One, Minus, etc.
There are several major services which support Linux directly in this way if you will to pay them, such as JustCloud, SugarSync, etc., though personally I find the free services to be much more interesting and useful for the sake of having multiple/redundant backups without it costing too many times the price of just having an awful lot of physical hard disks hanging around.
Fortunately for users of Linux, having a client program or application specific to each internet-based storage service is probably not as important as it is to users of the aforementioned mainstream OS's.
Linux users can simply choose services which offer access via the more standardized transfer protocols such as FTP, WebDAV, Swift, S3, etc., and then choose the cloud-storage providers that are accessible and usable in these ways. There is really no need for an OS-specific client program when the service offers/allows access through these avenues, because there are already so many FTP clients [for example] that are available for Linux.
That being said, it's no small wonder that there aren't more service-specific clients available for Linux. It can be intimidating indeed, from a software development point of view, to try to support Linux and BSD and other OS's like that because the distributions and flavors of Linux are so varied and often different in ways that can be subtle and/or obtuse. I think, though, that this kind of reason to avoid it is not well founded, especially considering that packages can be distributed or obtained in a "noarch" format that is specific to an OS or distro (i.e. Debian) but independent of any hardware types or architectures, as well as a "portable binary" kind of format that is specific to the hardware architecture but not constrained by any brand or distribution of Linux, as long as it is indeed Linux.
It seems to me that with those two options, it should be a relatively small challenge to make software for Linux that can be used with most any distribution.
Tuesday, March 18, 2014
"thread.error: can’t start new thread" workaround, CentOS yum, /etc/hosts.deny , echo \n and printf >> append file
BACKGROUND:
While observing network activity on a VPS/proxy server of mine, using the command
netstat -vT
to see a view of current connections and checking on recent failed or succeeded attempts to log in or log on to the system using commands like
cat /var/log/secure | grep 'Accepted password for root' | tail
and
cat /var/log/secure | grep 'Failed password ' | tail -n 20
and such,
I noticed what appeared to be someone running an automated script or macro or bot or program which was trying to gain access to the system by repeated logon attempts using what I assume someone thinks are the most commonly used usernames and passwords. I suppose this would be called hammering, or perhaps a remote brute-force password discovery attack of some kind.
Even with an automatic method like this, the chances of someone gaining access in this way were slim to none in this situation. My password is surely not on the list of "most commonly used passwords ever in the history of the Earth," unless it's an extremely long list that somehow includes the majority of passwords used in contemporary terrestrial history.
The offending party was only racking up about 1 attempt every 4 seconds, so yeah, there was not much chance at all of their hitting it blindly or randomly like that.
However nonthreatening it may have been, this way, it was still bothering me even if only because of how my logs were being filled up with garbage and becoming harder to read and monitor.
I got the idea that it should be a simple task to block the person's domain or IP address and preserve the simplicity of my security logs. I figured that this login service was probably operating with the TCP protocol or at least within a TCP wrapper, so I just navigated to the /etc folder and examined the "hosts.deny" file first.
I'm terrible with using "vi" to edit so I tried to edit it using "nano" instead. Apparently "nano" was not installed, I got one of those "command not found" errors. D'oh!
(I actually added the entries I'd wanted to into the "hosts.deny" file without using vi and without using nano or any other text editor, at this point. I'll describe that below but first I'll recount the tale of figuring out how to get nano in and working.)
Well this was a perturbing obstacle at first because I'd never actually bothered to check what operating system I was operating in with that system, so I had to determine which distribution of Linux I was using before I could use a package manager to install nano via the command line. While in the /etc directory, I used the command
ls | grep 'release'
which returned the line "redhat-release" , and upon using the "cat" command to see what tasty little tidbits of information were waiting inside that little file, it returned with
"CentOS release 5.9 (Final)"
which was very helpful in determining that the package management utility I'd want to use is called "yum," which is cute enough. ...Maybe too cute, that's an even cuter name for a command than "pacman" isn't it?
FOREGROUND:
I tried to install the package for "nano" using the command
yum install nano
but instead of the computer telling me how happy it was to have done what I'd wanted, it gave me some backtalk error like this:
"thread.error: can’t start new thread"
So my first thought was that I must be severely limited in the number of processes or threads I could run with my user account, which was a disappointing thought because I was logged in as the one and only "root" himself. I punched-in a command to have it tell me the maximum number of processes that I am limited to running at a time,
ps -L -u root | wc -l
which returned with "15," and that gave me pause because even though it seemed like a low number for that, I still couldn't really believe that "yum" would require more than one or two simultaneous processes just to install nano, even if it were simply a forking front-end for "rpm" or something.
[2015 edit: I believe the above command returns the number of currently-running processes rather than the result of actually probing for limitation]
I quickly found another page which explained that this error was [at least in this kind of situation] usually due to a lack of free memory, or available RAM, and not due to a limitation on the number of separate threads or processes that a user is allowed to be running at once.
The solution, I read, was to disable a plugin for yum called "fastestmirror" which is supposed to determine which mirror or which source of package files is the fastest one. There were several ways listed for disabling the plugin, and I chose to try the method which is to simply use an option in the command line, like so:
yum --disableplugin=fastestmirror update
After issuing this command, I found it was trying to update the entire OS or the entire collection of installed packages, which was not what I'd wanted. I decided to just let it finish, but it procured another error message and failed anyhow.
I tried again with the appropriate operation for installing nano, instead of the "update" operation, thusly:
yum --disableplugin=fastestmirror install nano
Sadly, that also failed to perform as I'd wished, but without terminating. It just stopped and never completed the command. After waiting long enough for me to determine that it was not going to finish, I simply logged in to the system remotely again with another terminal window and issued a "reboot" command. That worked splendidly for putting an end to the "yum" command, and I entertained hopes that it would clean up any mess left behind by yum's couple of difficulties.
After waiting for long enough to satisfy myself that it'd had enough time to reboot, I logged in again and issued the same command as shown above for disabling the "fastestmirror" plugin and installing "nano."
This time, it worked! I ran the nano program, one of my favorites, and was very pleased. I would have then continued to edit the "hosts.deny" file if not for the fact that I'd already added the badguys' IP to that file and successfully blocked the would-be intruder.
FLASHBACK:
To add entries to the "hosts.deny" file without a text editor I figured I'd just use a console command to print the appropriate line of text to the console but then redirect that console output to the end of the target file.
The line I wanted to add to "/etc/hosts.deny" [with the "\n" symbolizing an EOL/newline and with a funnier-sounding domain name than I was really blocking that day] is the following:
ALL: *.koreanbunghole.net\n
The "echo" command didn't seem to want to process the "\n" as a newline, when I was testing it without the redirect-append operator, so instead I used the "printf" command as follows (note this was all being done from within the "/etc" directory):
printf "ALL: *.koreanbunghole.net\n" >> hosts.deny
In order to refresh my memory and familiarize myself again with all the functions etc. being employed here, I first tried it on a test file called "testfile.txt.txt" instead of on the actual "hosts.deny" file a few times, and then used "cat" to see what was in the test file and confirm my expectations. Having to work without a text editor breeds caution, I guess.
To be honest: initially I'd neglected to include the asterisk and first period, and I found when I looked at the /var/log/secure file that they were still trying. I appended a second line using their actual IP address instead of the domain name, checked again, and sure enough I found that they had been stopped by that.
Tags: Failed password , thread.error: can’t start new thread , Accepted password for root , /var/log/secure , login attempts , yum , rpm , vi , nano , CentOS , redhat , release , Linux , VPS , ssh, tunneling , proxy , echo , printf , \n , newline , new line , EOL
While observing network activity on a VPS/proxy server of mine, using the command
netstat -vT
to see a view of current connections and checking on recent failed or succeeded attempts to log in or log on to the system using commands like
cat /var/log/secure | grep 'Accepted password for root' | tail
and
cat /var/log/secure | grep 'Failed password ' | tail -n 20
and such,
I noticed what appeared to be someone running an automated script or macro or bot or program which was trying to gain access to the system by repeated logon attempts using what I assume someone thinks are the most commonly used usernames and passwords. I suppose this would be called hammering, or perhaps a remote brute-force password discovery attack of some kind.
Even with an automatic method like this, the chances of someone gaining access in this way were slim to none in this situation. My password is surely not on the list of "most commonly used passwords ever in the history of the Earth," unless it's an extremely long list that somehow includes the majority of passwords used in contemporary terrestrial history.
The offending party was only racking up about 1 attempt every 4 seconds, so yeah, there was not much chance at all of their hitting it blindly or randomly like that.
However nonthreatening it may have been, this way, it was still bothering me even if only because of how my logs were being filled up with garbage and becoming harder to read and monitor.
I got the idea that it should be a simple task to block the person's domain or IP address and preserve the simplicity of my security logs. I figured that this login service was probably operating with the TCP protocol or at least within a TCP wrapper, so I just navigated to the /etc folder and examined the "hosts.deny" file first.
I'm terrible with using "vi" to edit so I tried to edit it using "nano" instead. Apparently "nano" was not installed, I got one of those "command not found" errors. D'oh!
(I actually added the entries I'd wanted to into the "hosts.deny" file without using vi and without using nano or any other text editor, at this point. I'll describe that below but first I'll recount the tale of figuring out how to get nano in and working.)
Well this was a perturbing obstacle at first because I'd never actually bothered to check what operating system I was operating in with that system, so I had to determine which distribution of Linux I was using before I could use a package manager to install nano via the command line. While in the /etc directory, I used the command
ls | grep 'release'
which returned the line "redhat-release" , and upon using the "cat" command to see what tasty little tidbits of information were waiting inside that little file, it returned with
"CentOS release 5.9 (Final)"
which was very helpful in determining that the package management utility I'd want to use is called "yum," which is cute enough. ...Maybe too cute, that's an even cuter name for a command than "pacman" isn't it?
FOREGROUND:
I tried to install the package for "nano" using the command
yum install nano
but instead of the computer telling me how happy it was to have done what I'd wanted, it gave me some backtalk error like this:
"thread.error: can’t start new thread"
So my first thought was that I must be severely limited in the number of processes or threads I could run with my user account, which was a disappointing thought because I was logged in as the one and only "root" himself. I punched-in a command to have it tell me the maximum number of processes that I am limited to running at a time,
ps -L -u root | wc -l
which returned with "15," and that gave me pause because even though it seemed like a low number for that, I still couldn't really believe that "yum" would require more than one or two simultaneous processes just to install nano, even if it were simply a forking front-end for "rpm" or something.
[2015 edit: I believe the above command returns the number of currently-running processes rather than the result of actually probing for limitation]
I quickly found another page which explained that this error was [at least in this kind of situation] usually due to a lack of free memory, or available RAM, and not due to a limitation on the number of separate threads or processes that a user is allowed to be running at once.
The solution, I read, was to disable a plugin for yum called "fastestmirror" which is supposed to determine which mirror or which source of package files is the fastest one. There were several ways listed for disabling the plugin, and I chose to try the method which is to simply use an option in the command line, like so:
yum --disableplugin=fastestmirror update
After issuing this command, I found it was trying to update the entire OS or the entire collection of installed packages, which was not what I'd wanted. I decided to just let it finish, but it procured another error message and failed anyhow.
I tried again with the appropriate operation for installing nano, instead of the "update" operation, thusly:
yum --disableplugin=fastestmirror install nano
Sadly, that also failed to perform as I'd wished, but without terminating. It just stopped and never completed the command. After waiting long enough for me to determine that it was not going to finish, I simply logged in to the system remotely again with another terminal window and issued a "reboot" command. That worked splendidly for putting an end to the "yum" command, and I entertained hopes that it would clean up any mess left behind by yum's couple of difficulties.
After waiting for long enough to satisfy myself that it'd had enough time to reboot, I logged in again and issued the same command as shown above for disabling the "fastestmirror" plugin and installing "nano."
This time, it worked! I ran the nano program, one of my favorites, and was very pleased. I would have then continued to edit the "hosts.deny" file if not for the fact that I'd already added the badguys' IP to that file and successfully blocked the would-be intruder.
FLASHBACK:
To add entries to the "hosts.deny" file without a text editor I figured I'd just use a console command to print the appropriate line of text to the console but then redirect that console output to the end of the target file.
The line I wanted to add to "/etc/hosts.deny" [with the "\n" symbolizing an EOL/newline and with a funnier-sounding domain name than I was really blocking that day] is the following:
ALL: *.koreanbunghole.net\n
The "echo" command didn't seem to want to process the "\n" as a newline, when I was testing it without the redirect-append operator, so instead I used the "printf" command as follows (note this was all being done from within the "/etc" directory):
printf "ALL: *.koreanbunghole.net\n" >> hosts.deny
In order to refresh my memory and familiarize myself again with all the functions etc. being employed here, I first tried it on a test file called "testfile.txt.txt" instead of on the actual "hosts.deny" file a few times, and then used "cat" to see what was in the test file and confirm my expectations. Having to work without a text editor breeds caution, I guess.
To be honest: initially I'd neglected to include the asterisk and first period, and I found when I looked at the /var/log/secure file that they were still trying. I appended a second line using their actual IP address instead of the domain name, checked again, and sure enough I found that they had been stopped by that.
Tags: Failed password , thread.error: can’t start new thread , Accepted password for root , /var/log/secure , login attempts , yum , rpm , vi , nano , CentOS , redhat , release , Linux , VPS , ssh, tunneling , proxy , echo , printf , \n , newline , new line , EOL
Subscribe to:
Comments (Atom)