Well It's been a crazy couple of months, but I am back!
Anywho, something cool I learned the other day.
First off the motivation behind it. Say you've got a server which is _always_ accepting connections. Now say you would like to kill the server in a graceful manner which allows you to update and start a new instance of the server in a few seconds. You could try to reject all connections on 'graceful' shutdown, however there is a problem. Even if you close the socket, clear the queue, and reject all previous connections, when you close a connection in TCP, it throws it into the TIME_WAIT state because of the nature of TCP.
The reason for going to TIME_WAIT is because TCP is not supposed to drop messages. Say we were talking and then I said goodbye. In real life you would also say goodbye before we parted ways, but in TCP there is no acknowledgment of the end of the session because to say 'goodbye' the server closes the socket. However, what if the client never receives the packet which tells it the connection is closed at the other end? It would then attempt to request any dropped packets in 30seconds - 4 minutes later. This is what the TIME_WAIT state is for, it allows the client to request dropped packets.
Thats all well and good, but I don't want to wait for this timeout, I want to rebind my socket NOW! Well there is a way around it. You can use the function setsocketoption to set SO_REUSEPORT which if set, allows future instances to reopen a socket in a TIME_WAIT state. This means that if i kill a server with this option set (cleanly or with a signal) the socket will go to TIME_WAIT, but with the option set, we can reopen it!
I'm not sure what the last two vars are for, but the example i found passes 'one = 1;' in which i assume is just a 'dont do anything' type of deal, and like many socket functions you also have to pass the size of the var in with it.
setsockopt(serversock, SOL_SOCKET, SO_REUSEADDR, &one, sizeof(one);
works like a charm on suse 10.2, my code for work rebinds instantly, ill update this with other portability notes (specifically rhel4 and Mac OS X).
Enjoy,
-Crusher4
Friday, June 22, 2007
Friday, May 4, 2007
Time is everything...again.
Alright this post has nothing to do with tehcnology or coding except for it is the reason why I have been unable to post anything of value in the last two weeks...too busy coding. Now that finals and 360 project are done, I should have time to post things again!
But now. Now is for sleep.
-Crusher4
But now. Now is for sleep.
-Crusher4
Tuesday, April 10, 2007
Time is everything.
A simple one for you today, but useful none the less.
It involves time. Whether or not it is the fire in which we burn is up for grabs, but you still might want your libc program to be able to output the time!
In a program I wrote this week I did it this way:
#include <time.h>
char* mtime()
{
static char return[16];
time_t rawtime;
struct tm * t;
time( &rawtime );
t=localtime( &rawtime );
snprintf(return,16,"[%02d:%02d%02d]",t->tm_hr,t->tm_min,t->tm_sec);
}
This function returns a string in the format [hr:mn:sc] (hour:minute:second). It will always print two characters for each field, if none are provided it will print zeroes (this is useful as time returns 1 second as 1 not 01, so if you want the string to always be the same length you have to pad it with zeroes).
Time fills the time_t variable called rawtime with the current time since 1970 in seconds. To get something useful to humans we use another function on that returned time. In this case since we want the local time (which also includes the timezone calculation for us) then we call rawtime, which returns a struct* to a tm. I assume it is statically allocated within the localtime function. It may be wise to copy the data out of the struct if you intend to use it much later in your code, but since I use it on the next line, and I haven’t had any problems with it, my code will stay like that until there is a problem :).
Enjoy,
-Crusher4
It involves time. Whether or not it is the fire in which we burn is up for grabs, but you still might want your libc program to be able to output the time!
In a program I wrote this week I did it this way:
#include <time.h>
char* mtime()
{
static char return[16];
time_t rawtime;
struct tm * t;
time( &rawtime );
t=localtime( &rawtime );
snprintf(return,16,"[%02d:%02d%02d]",t->tm_hr,t->tm_min,t->tm_sec);
}
This function returns a string in the format [hr:mn:sc] (hour:minute:second). It will always print two characters for each field, if none are provided it will print zeroes (this is useful as time returns 1 second as 1 not 01, so if you want the string to always be the same length you have to pad it with zeroes).
Time fills the time_t variable called rawtime with the current time since 1970 in seconds. To get something useful to humans we use another function on that returned time. In this case since we want the local time (which also includes the timezone calculation for us) then we call rawtime, which returns a struct* to a tm. I assume it is statically allocated within the localtime function. It may be wise to copy the data out of the struct if you intend to use it much later in your code, but since I use it on the next line, and I haven’t had any problems with it, my code will stay like that until there is a problem :).
Enjoy,
-Crusher4
Monday, April 2, 2007
pthread attributes.
I meant to post more last week, but we had a huge LAN party at WSU a few days ago, thanks to everyone who came and made it awesome :).
Anywho...My next tidbit of information is about pthreads. I had to learn this because when I was writing a piece of software for work, I noticed that I couldn't launch more than about 300 threads before it would start refusing to launch more. After a few hours of research on google I found that it was due to the fact that even though each thread was only using kilobytes of memory, the default thread stack size is somewhere in the range of 1-8mb in most linuxes (it was 8 in the os I was using), which means that even though there was only about 4mb allocated in my program the system was "out of memory" to allocate. But thats ok, we can fix it, we have the technology.
This can all be remedied by changing the thread stack size before we create new pthreads. It is accomplished with this simple bit of code:
size_t size = 0x10000;
pthread_attr_t tattr;
if (pthread_attr_init(&tattr) !=0 )
{fprintf(stderr,"pthread_attr_init() failed\n");}
if (pthread_attr_setstacksize(&tattr,size) !=0 )
{fprintf(stderr,"pthread_attr_stacksize() failed\n");}
pthread_attr_getstacksize(&tattr,&size); /* find out the stacksize */
fprintf(stderr,"Thread stack size: %dKB\n",(int)size/1024);
The last two lines are not needed, but I like to keep them in especially while I am developing this software, to make sure it is working the same way on various computers I try compiling it for. Using that value of size it will have threads with 64KB of stack size.
You can check the defaults of your linux for stack size, open file discriptors, and all the other things you need to make sure of when writing an intensive network app by typing 'ulimit -a'.
But more ulimit things later.
Over and out.
-Crusher4
Anywho...My next tidbit of information is about pthreads. I had to learn this because when I was writing a piece of software for work, I noticed that I couldn't launch more than about 300 threads before it would start refusing to launch more. After a few hours of research on google I found that it was due to the fact that even though each thread was only using kilobytes of memory, the default thread stack size is somewhere in the range of 1-8mb in most linuxes (it was 8 in the os I was using), which means that even though there was only about 4mb allocated in my program the system was "out of memory" to allocate. But thats ok, we can fix it, we have the technology.
This can all be remedied by changing the thread stack size before we create new pthreads. It is accomplished with this simple bit of code:
size_t size = 0x10000;
pthread_attr_t tattr;
if (pthread_attr_init(&tattr) !=0 )
{fprintf(stderr,"pthread_attr_init() failed\n");}
if (pthread_attr_setstacksize(&tattr,size) !=0 )
{fprintf(stderr,"pthread_attr_stacksize() failed\n");}
pthread_attr_getstacksize(&tattr,&size); /* find out the stacksize */
fprintf(stderr,"Thread stack size: %dKB\n",(int)size/1024);
The last two lines are not needed, but I like to keep them in especially while I am developing this software, to make sure it is working the same way on various computers I try compiling it for. Using that value of size it will have threads with 64KB of stack size.
You can check the defaults of your linux for stack size, open file discriptors, and all the other things you need to make sure of when writing an intensive network app by typing 'ulimit -a'.
But more ulimit things later.
Over and out.
-Crusher4
Tuesday, March 27, 2007
signals (eg SIGTERM, SIGPIPE, SIGHUP, SIGINT)
First off, something a bit Off Topic. Why is it that all blog editors have to be so tiny. I have a maximized 1280x1024 window and the blog editor is about half of that. *sigh*
Anywho, todays topic: unix/linux signals. I had to introduce myself to them for a recent project at work where a wrote a threaded socket-ed server. I had used send and recv because they allowed for "MSG_NOSIGNAL" at least until I got home to my Mac OS X computer, which apparently doesn't have that flag (or if it does its not #defined the same way). At any rate its just as easy to handle the signal that is raised by a broken/disconnected socket, which is SIGPIPE (the broken pipe signal).
There are two (old) ways to handle such problems. The first way I learned was:
#include <sys/signal.h>
static void signalHandler() { //handle the signal somehow; }
int main()
{
// blah
signal(SIGPIPE,signalHandler);
//blah
}
This is a simple way to handle any signal and deal with it. The problem with this method is that if you are just ignoring the signal, you have to re register the signal handler every time it is used. One solution to the "only registered once" problem is to update our signal handler so it looks like this:
static void signalHandler()
{
//handle the signal somehow;
signal(SIGPIPE,signalHandler);
}
This is a good way to handle a reoccurring problem where you actually want to fix something and then handle it again. Obviously you don't have to do this for something you only want to handle once, such as say a SIGTERM, where you just want to close your sockets before exiting.
This is obviously much more than one might want to put in simply to ignore a signal however, which is why there is a function called:
sigignore(SIGPIPE);
This will ignore a signal to your program until the program quits.
I like this way of handling the SIGPIPE raised by a socket failure. Its not bad coding practice because sockets return either 0 or -1 on failure of a read/write, which is better anyway, because signal sending is only "supported" on a per application level. It might work and send it to the correct thread, but it is NOT guaranteed. Therefor you can only count on your main program to recieve the signals anyway. So by ignoring the signal (which would normally make your program take a crap) and checking the status of the socket on every send/recv (or read/write), your program has a good idea of what is going on.
Some common signals are:
Catchable:
SIGTERM = sent by kill among others
SIGPIPE = broken pipe, sent by dead sockets, pipes, etc.
SIGHUP = "hang up" sent by exiting the shell the process was running in etc.
SIGIO = configurable exception raised on sockets etc, when something is ready to happen (data recieved etc.)
SIGINT = "interrupt" sent by ctrl - c among other things.
Not able to be caught:
SIGKILL = sent by kill -9
There are more signals. Something like 22 I think, but those are the only ones I have run across so far, and I have only actually handled SIGTERM, SIGINT, and SIGPIPE. I like to handle SIGTERM and SIGINT so I can close my sockets before exiting
because it makes the program start quicker next time when it doesn't have to wait for previously bound sockets to time out before rebinding.
The last thing to note (sadly) is that this signal, sigignore, and all the other functions related in the man pages, are actually depricated. Thats not to say that they will be removed from any linux anytime soon. The new method is really overkill for anything I was doing, so I didn't bother to learn it. I hope that this method is never removed, because signal and sigignore are super easy ways to get things done. Im not disagreeing with "better" functionality, one of the features added in the new interface was ability to have a "previous" handler stored, which you could switch back to when you were done. Once again this is massive overkill for what I need to do, and in most cases, this will work just fine for years (I'm sure) to come.
Happy coding!
-Crusher4
Anywho, todays topic: unix/linux signals. I had to introduce myself to them for a recent project at work where a wrote a threaded socket-ed server. I had used send and recv because they allowed for "MSG_NOSIGNAL" at least until I got home to my Mac OS X computer, which apparently doesn't have that flag (or if it does its not #defined the same way). At any rate its just as easy to handle the signal that is raised by a broken/disconnected socket, which is SIGPIPE (the broken pipe signal).
There are two (old) ways to handle such problems. The first way I learned was:
#include <sys/signal.h>
static void signalHandler() { //handle the signal somehow; }
int main()
{
// blah
signal(SIGPIPE,signalHandler);
//blah
}
This is a simple way to handle any signal and deal with it. The problem with this method is that if you are just ignoring the signal, you have to re register the signal handler every time it is used. One solution to the "only registered once" problem is to update our signal handler so it looks like this:
static void signalHandler()
{
//handle the signal somehow;
signal(SIGPIPE,signalHandler);
}
This is a good way to handle a reoccurring problem where you actually want to fix something and then handle it again. Obviously you don't have to do this for something you only want to handle once, such as say a SIGTERM, where you just want to close your sockets before exiting.
This is obviously much more than one might want to put in simply to ignore a signal however, which is why there is a function called:
sigignore(SIGPIPE);
This will ignore a signal to your program until the program quits.
I like this way of handling the SIGPIPE raised by a socket failure. Its not bad coding practice because sockets return either 0 or -1 on failure of a read/write, which is better anyway, because signal sending is only "supported" on a per application level. It might work and send it to the correct thread, but it is NOT guaranteed. Therefor you can only count on your main program to recieve the signals anyway. So by ignoring the signal (which would normally make your program take a crap) and checking the status of the socket on every send/recv (or read/write), your program has a good idea of what is going on.
Some common signals are:
Catchable:
SIGTERM = sent by kill among others
SIGPIPE = broken pipe, sent by dead sockets, pipes, etc.
SIGHUP = "hang up" sent by exiting the shell the process was running in etc.
SIGIO = configurable exception raised on sockets etc, when something is ready to happen (data recieved etc.)
SIGINT = "interrupt" sent by ctrl - c among other things.
Not able to be caught:
SIGKILL = sent by kill -9
There are more signals. Something like 22 I think, but those are the only ones I have run across so far, and I have only actually handled SIGTERM, SIGINT, and SIGPIPE. I like to handle SIGTERM and SIGINT so I can close my sockets before exiting
because it makes the program start quicker next time when it doesn't have to wait for previously bound sockets to time out before rebinding.
The last thing to note (sadly) is that this signal, sigignore, and all the other functions related in the man pages, are actually depricated. Thats not to say that they will be removed from any linux anytime soon. The new method is really overkill for anything I was doing, so I didn't bother to learn it. I hope that this method is never removed, because signal and sigignore are super easy ways to get things done. Im not disagreeing with "better" functionality, one of the features added in the new interface was ability to have a "previous" handler stored, which you could switch back to when you were done. Once again this is massive overkill for what I need to do, and in most cases, this will work just fine for years (I'm sure) to come.
Happy coding!
-Crusher4
Monday, March 26, 2007
Introduction
Well. This is my first "Tech Blog" post, so I suppose I'll give you some background about what this is all about.
As opposed to my other blogs which are about my "life" this blog I will try to keep on the topic of tech savvy things at the least, but at the moment my current focus are programs which can be compiled with gcc. That means my blog title should probably include Windows, but as of yet I've been to lazy to get mingw / mingcc working on my windows environment.
If you see any "tech bias" here it will probably be towards the mac/linux side...BUT, thats mostly because the shell environment in windows is pretty bleh compared to the alternatives. Don't get me wrong, I've taken plenty of windows IT classes.
Speaking of classes: I'm a Computer Science major at Washington State University. I came to WSU in 2002. I was here for a year and a half before I "left" due to academic deficiency (which means kicked out by the way). After a year and a half of Spokane Falls Community College aka SFCC (you are required to leave for a minimum of a year due to academic deficiency) I had quite a new outlook. Looking back on it, I'm glad I got the boot, because it allowed me to get an _ENTIRELY_ new perspective on a lot of things. For my first quarter at SFCC I took IS/IT classes which were quite enlightening and allowed me to learn a lot about windows in a domain environment. The real treat was when I started taking CS classes at SFCC. My teacher through my next 3 quarters of CS at SFCC was John Mill -- who is without a doubt the best computer science teacher I have ever had. I thought I knew quite a bit going in there, but anything that had hurt my brain to think about before SFCC like Binary Search Trees / Advanced Sorting and other topics, all really came together, which helped me to become a coder that likes to carefully consider his approaches before delving headlong into a project. I'm not saying I'm "uberfantastic", But I try to be the best I can. Anywho, going to SFCC really helped to cement the style of thinking that I needed to solve problems with code, without feeling overwhelmed. If there is any skill a coder needs, its not to be overwhelmed...you've just got to loop through the steps to solve the problem after all :)
So anyway. I'm currently in CS 360 422 and 317. To anyone who knows the WSU curriculum that means I'm in one real coding class and two pretty fluffy classes. 317 is my least favorite because its all about things like set theory, automata, etc. Things that make sense to me in practice, but not in theory. 422 is all about "coding in the real world" about testing, etc. Things that you'll have to deal with if you work at places like M$. Not something I look forward to, but not utterly impossible either, and its a pretty fun class anyway. 360 is my favorite class. It's called "operating systems" although its more like "low level coding in c in relation to linx and ext2fs". Anyway you slice it, its superfun, because every time you finish a lab for that class you feel accomplished. And I learn a lot of low level stuff that seems simple after the fact, but to learn on my own would be daunting.
I also have two "tech" jobs. One for student computing...which is basically an IT position, management, fixing, etc of computers -- however since I was just hired this semester, most of my job is sitting swiping people into labs and answering questions...but it still beats the hell out of the 4 years I worked at subway. ( I'm nearly 23 btw -- class of 2002 ). My other job has its IT elements, but is basically a "huge" graduate lab. I mean huge in the sense of how much coding they accomplish/research/etc. the actual "lab" is only about 16 computers. Its been really fun. I started out by re assembling the lab as best as I could. That having been mostly accomplished a couple weeks ago, my attentions turned to my first assignment, which happened to be writing a one way TCP proxy.
Using the knowledge of sockets I gained in 360 writing the initial code skeleton wasnt hard...but doing all the linux tricks that make a program "slick" has taken me days to learn. So over the course of the next few days I'll be discussing these low level linux signal / socket / thread / etc tricks and basic knowledge that I've learned.
So stay tuned :)
(sorry for the boring intro)
-Crusher4
As opposed to my other blogs which are about my "life" this blog I will try to keep on the topic of tech savvy things at the least, but at the moment my current focus are programs which can be compiled with gcc. That means my blog title should probably include Windows, but as of yet I've been to lazy to get mingw / mingcc working on my windows environment.
If you see any "tech bias" here it will probably be towards the mac/linux side...BUT, thats mostly because the shell environment in windows is pretty bleh compared to the alternatives. Don't get me wrong, I've taken plenty of windows IT classes.
Speaking of classes: I'm a Computer Science major at Washington State University. I came to WSU in 2002. I was here for a year and a half before I "left" due to academic deficiency (which means kicked out by the way). After a year and a half of Spokane Falls Community College aka SFCC (you are required to leave for a minimum of a year due to academic deficiency) I had quite a new outlook. Looking back on it, I'm glad I got the boot, because it allowed me to get an _ENTIRELY_ new perspective on a lot of things. For my first quarter at SFCC I took IS/IT classes which were quite enlightening and allowed me to learn a lot about windows in a domain environment. The real treat was when I started taking CS classes at SFCC. My teacher through my next 3 quarters of CS at SFCC was John Mill -- who is without a doubt the best computer science teacher I have ever had. I thought I knew quite a bit going in there, but anything that had hurt my brain to think about before SFCC like Binary Search Trees / Advanced Sorting and other topics, all really came together, which helped me to become a coder that likes to carefully consider his approaches before delving headlong into a project. I'm not saying I'm "uberfantastic", But I try to be the best I can. Anywho, going to SFCC really helped to cement the style of thinking that I needed to solve problems with code, without feeling overwhelmed. If there is any skill a coder needs, its not to be overwhelmed...you've just got to loop through the steps to solve the problem after all :)
So anyway. I'm currently in CS 360 422 and 317. To anyone who knows the WSU curriculum that means I'm in one real coding class and two pretty fluffy classes. 317 is my least favorite because its all about things like set theory, automata, etc. Things that make sense to me in practice, but not in theory. 422 is all about "coding in the real world" about testing, etc. Things that you'll have to deal with if you work at places like M$. Not something I look forward to, but not utterly impossible either, and its a pretty fun class anyway. 360 is my favorite class. It's called "operating systems" although its more like "low level coding in c in relation to linx and ext2fs". Anyway you slice it, its superfun, because every time you finish a lab for that class you feel accomplished. And I learn a lot of low level stuff that seems simple after the fact, but to learn on my own would be daunting.
I also have two "tech" jobs. One for student computing...which is basically an IT position, management, fixing, etc of computers -- however since I was just hired this semester, most of my job is sitting swiping people into labs and answering questions...but it still beats the hell out of the 4 years I worked at subway. ( I'm nearly 23 btw -- class of 2002 ). My other job has its IT elements, but is basically a "huge" graduate lab. I mean huge in the sense of how much coding they accomplish/research/etc. the actual "lab" is only about 16 computers. Its been really fun. I started out by re assembling the lab as best as I could. That having been mostly accomplished a couple weeks ago, my attentions turned to my first assignment, which happened to be writing a one way TCP proxy.
Using the knowledge of sockets I gained in 360 writing the initial code skeleton wasnt hard...but doing all the linux tricks that make a program "slick" has taken me days to learn. So over the course of the next few days I'll be discussing these low level linux signal / socket / thread / etc tricks and basic knowledge that I've learned.
So stay tuned :)
(sorry for the boring intro)
-Crusher4
Subscribe to:
Posts (Atom)