Running something "all tihe time" (as a normal user)

 N.B. the following is not a replacement for proper services, the use-case is slightly different.

So, you have an unstable application. But you want it to be running all the time, even when you are not there to restart it after a crash (or a reboot).

You could use cron, but that will start a new instance of the app every single time. Hardly ideal!

Well actually the solution IS to use cron but not to start the app directly.

Below is a small script I wrote which runs a whole bunch of applications in the background and automagically restarts them if the crash.
Put small scripts into the ~/screens folder and this script will make sure each of them is running, and start it if not.

ls ~/screens -l  | grep "^...x" | sed -r "s/.*?:[0-9][0-9]\ //"  | sed -r "s/.*?[A-Za-z]* *[0-9]* *[0-9]{4} //"|  grep -v "^~" | grep -v "^#" | ( while read f
do
        f=~/screens/$f
        echo $f
        name=AS_$(cat $f | grep -i "^[ \t]*#[ \t]*name:" | sed "s/[^:]*:[ \t]*//")
        desc=$(cat $f | grep -i "^[ \t]*#[ \t]*desc(ription)?:" | sed "s/[^:]*:[ \t]*//")

        echo -en "Checking $desc ($name) ... "
        if [ $( screen -ls | grep -i $name | wc -l)  == 1 ]
        then
                echo "already running"
        else
                echo -en " starting ..."
                screen -dmS $name -s $f && echo "done" || echo "failed"
        fi
done
)

Set this script to run as often as you like (I have mine set at 5 minutes) using crontab
#crontab -e
*/5 * * * * /home/lee/screens2.sh >/dev/null 2>/dev/null







Job Done!

Port forwarding with ssh

So I've had this all working previously but I had forgotten all the hoops that we're necessary to get it to work.  

 N.B. all of the below assumes that a "proper" vpn solution is not an option

The aim:

To forward a port on network A to a machine on network B in such a way as to be accessible by all machines on network A, using the address that the machine on network B has.

The Caveat:

It is not possible to make an ssh connection from Network A into Network B due to the the firewall setup, however the reverse is possible.  

The solution:

So ssh has this handy -R flag which allows you to forward a port from the remote host to the local network. Setting up a secondary ip address is easy enough using Linux's config tools (yast in my case), ifconfig or ip* The final part of the puzzle is to set up a route from all relevant machines on network A that direct it to the ssh gateway machine.  

The problem:

ssh, by default, does not allow you to specify the remote address to bind to (or rather, you can specify it but the packets will be silently dropped with no error message to indicate what you did wrong).

The Fix:

edit /etc/ssh/sshd_config and add the line

GatewayPorts yes

While you're there I recommend you also add

TCPKeepAlive yes

Why these two options are not enabled by default is beyond me.

The complete solution:

ok, so you have the following setup:
Network A: 192.168.0.0/24
Computer 1: 192.168.0.2, Windows
Computer 2: 192.168.0.3, *nix
Computer 3: 192.168.0.4, *nix, used as gateway
Router 1: internal: 192.168.0.1, external: 10.0.0.1, external port 22 forwarded to 192.168.0.4


Network B: 192.168.1.0/24

Computer 4: 192.168.1.2, Target (service port: 1234)
Computer 5: 192.168.1.3, *nix, used as gateway


Router 2: internal: 192.168.1.1, external: 10.0.0.2

On Computer 3:

Edit (as root) /etc/ssh/sshd_config in your favourite editor and add the following options:

GatewayPorts yes
TCPKeepAlive yes

Use your systems config tools to add a second ip address for the network interface [in yast this is Network Devices->Network Settings->Edit->Add]
Set the ip address to be the same as the target computer (e.g. 192.168.1.2).
Don't worry about it being in a different ip range - we'll fix that later.

Create a new user and create an ssh private key for that user

On Computer 5:

Run the following command as a normal user (or get someone to run it for you): **

#ssh -v -R 192.168.1.2:1234:192.168.1.2:1234 user@10.0.0.1

On Computers 1 & 2:

Configure a route to access the remote computer
network: 192.168.1.0
via: 192.168.0.4

* I only discovered ip existed today. It seems I am not the only one. Spread the word!
** See a separate blog post to follow about how to have this running all the time.

Viewing print styles with firebug

After much searching I couldn't find a way to reliably view the css print styles in the browser and debug them using firebug. So I decided to make my own solution. Save the following link as a bookmark called "Toggle Print Styles" and behold the magic. This link has only been tested in firefox but it ought to be cross-browser. javascript:%20(function(){%20%20%20function%20ShowPrintStyles(Style)%20%20%20{%20%20%20%20%20var%20$Style=$(Style);%20%20%20%20%20$Style.text($Style.text().replace('@media%20screen',%20'@media%20xxscreen').replace('@media%20print',%20'@media%20screen'));%20%20%20%20%20$Style.attr('ShowingPrintStyles',%20true);%20%20%20%20%20console.log(Style);%20%20%20}%20%20%20ShowHide%20=%20window.ShowPrintStyles;%20%20%20if%20(ShowHide)%20%20%20{%20%20%20%20%20window.ShowPrintStyles=false;%20%20%20%20%20$('style[OrigSrc]').each(function(id,%20item){%20%20%20%20%20%20%20var%20$Link%20=%20$('<link>').attr({rel:'stylesheet',%20href:$(item).attr('OrigSrc')});%20%20%20%20%20%20%20$(item).replaceWith($Link);%20%20%20%20%20});%20%20%20}%20else%20{%20%20%20%20%20window.ShowPrintStyles=true;%20%20%20%20%20$('link').each(function(id,%20item){%20%20%20%20%20%20%20var%20$Style%20=%20$('<style>').attr({OrigSrc:$(item).attr('href')}).load($(item).attr('href'),%20null,%20function(){%20%20%20%20%20%20%20%20%20ShowPrintStyles($Style[0]);%20%20%20%20%20%20%20%20%20$(item).replaceWith($Style);%20%20%20%20%20%20%20});%20%20%20%20%20});%20%20%20}%20})(); Let's take a look at that link in detail. It is actually a javascript link. I had some issues with a multi-statement javascript: url in firefox so I wrapped the whole thing in a closure so the link is actually a single statement. javascript: (function(){ function ShowPrintStyles(Style) { var $Style=$(Style); $Style.text($Style.text().replace('@media screen', '@media xxscreen').replace('@media print', '@media screen')); $Style.attr('ShowingPrintStyles', true); console.log(Style); } ShowHide = window.ShowPrintStyles; if (ShowHide) { window.ShowPrintStyles=false; $('style[OrigSrc]').each(function(id, item){ var $Link = $('<link>').attr({rel:'stylesheet', href:$(item).attr('OrigSrc')}); $(item).replaceWith($Link); }); } else { window.ShowPrintStyles=true; $('link').each(function(id, item){ var $Style = $('<style>').attr({OrigSrc:$(item).attr('href')}) .load($(item).attr('href'), null, function(){ ShowPrintStyles($Style[0]); $(item).replaceWith($Style); }); }); } })(); The ShowPrintStyles function takes an inline style element, replaces any "@media screen" sections with "@media xxscreen", then does likewise for the print sections to screen. This obviously has the effect of disabling any screen-only styles, while enabling the print-only ones (Assuming you are viewing on a screen that is - if not then adjust the javascript to suit your device) A property is set on the global window object to indicate whether we are showing or hiding the print styles. If we are showing then any link elements are replaced with an inline style so that we can modify the css content. If we are hiding then the inline styles are replaced with their original link elements.

Private members in javascript classes

Whoever told you this couldn't be done was lying.

By utilising closures we can create a class that holds its own private members which cannot be accessed outside of that class - even by adding a new function onto the class

Consider the following javascript function

function Outer() { var PrivateVar='PrivateValue'; function Inner() { PrivateVar='Some Value'; } }

This is known as a closure.
You can read more about them here: http://stackoverflow.com/questions/111102/how-do-javascript-closures-work

Put simply the inner function has access to the variables in the outer function.
These variables are "live" - that is they are not fixed when the inner function was created.

Now consider that the Outer and Inner functions are, in fact, constructors

function MyClassWithPrivateVars(params) { var PrivateVar; function MyClass(params) { PrivateVar='Initial Value'; } return new MyClass(params); } var Instance = new MyClassWithPrivateVars('whatever');

When you call the outer constructor you are actually calling the inner constructor.
The inner constructor has access to the vars within the outer constructor as before.

Let's add a method to the inner constructor.
This new method of course has access to the variables in the outer constructor.

In this instance the method is a "property".

function MyClassWithPrivateVars(params) { var PrivateVar; function MyClass(params) { PrivateVar='Initial Value'; this.PublicProperty = function(value) { if (typeof(value)!=="undefined") { PrivateVar=value; } return PrivateVar; } } return new MyClass(); }

Now if we create an instance of MyClassWithPrivateVars and invoke the property we'll see that it gets the value as expected.

var MyInstance=new MyClassWithPrivateVars(); console.log(MyInstance.PublicProperty());

Outputs "Initial Value" because the private variable was set to that value in the constructor.

MyInstance.PublicProperty(10); console.log(MyInstance.PublicProperty());

This time we get 10

So is the variable really private?

Well - let's try and access it directly:

console.log(MyInstance.PrivateVar);

This should output undefined as PrivateVar is not a member of the inner class.

Likewise

console.log(MyClassWithPrivateVars.PrivateVar);

is also undefined because PrivateVar is a member of the instance of the outer class - not of the constructor itself.

Type 'System.Collections.Generic.IDictionary`2[[System.String, mscorlib, Version=4.0.0.0, Culture=neutral, PublicKeyToken=b77a5c561934e089],[System.Object, mscorlib, Version=4.0.0.0, Culture=neutral, PublicKeyToken=b77a5c561934e089]]' is not supported for deserialization of an array.

So I was getting this error when calling a page method from jquery. Google turned up quite a few variations but not the exact string, and the suggestions associated with those variations were not relevant. It turned out that in my case the data packet I was sending in the jquery ajax call was wrong. I had code like this:

MyParam = []; ... $.ajax({ type: 'POST', url: document.location.pathname + '/DoSomething', data: JSON.stringify(MyParam), contentType: "application/json; charset=utf-8", dataType: "json", complete: function (a, b, c, d) { console.log(a, b, c, d); } });

and my page method looked like this:

[WebMethod()] public static void DoSomething(List<s> B) { B.ToString(); }

fairly simple but there are 2 things wrong with this. The first problem was the data - I hadn't named the paramater

so the fixed javascript code looks like this:

MyParam = []; ... $.ajax({ type: 'POST', url: document.location.pathname + '/DoSomething', data: JSON.stringify({B:MyParam}), contentType: "application/json; charset=utf-8", dataType: "json", complete: function (a, b, c, d) { console.log(a, b, c, d); } });

The second problem was it turns out the .NET javascript seraliser doesn't like List<t> so a quick change to the C# fixed that.

[WebMethod()] public static void DoSomething(S[] B) { B.ToString(); }

So in summary - make sure you pass an object with named parameters as the data argument - not simple the content of the only parameter!

Staging long loops in Javascript

So I've encounted this issue a number of times where you need to do something, probably in a loop, which is going to take some time and the browser will "hang" will the loop takes place.
Sometimes you may even get a message pop up asking if you want to terminate a stuck script.

Not only does this make for a bad user experience but it can cause scripts to terminate without warning.

 The solution?
Plenty of posts on forum sites such as StackOverflow will tell you to use setTimeout.
It can get kind of messy though - so I thought I'd write a wrapper function to take away the pain, and have something that *looks* mostly like the familiar simple loop.
Now obviously there is a performance hit by adding all these extra "unnecessary" delays. (approx. 30% for a tight loop, less for a complex one) BUT if the browser pops up a box and waits for the user to respond, this will add much more delay to the total execution time!
so - heres the code:
function StagedLoop(IterationsBetweenPauses, Delay, AtLeastOneDelay, Test, Run, Complete) { /// <summary>Stages a loop, inserting timer pauses every n Iterations ///Warning - this method is asyncronus and will exit before looping is completed! /// put code to be run only after looping has finished into the Complete inline function parameter ///</summary> ///<param name="Test">A method to run which will perform the "test" part of the loop to see if looping should continue. This test will be run at the start of the loop (i.e. the loop may run 0 or more times)</param> ///<param name="Run">A method which will be run on every iteration of the loop - don't forget to increment any counters!</param> ///<param name="IterationsBetweenPauses">How many loops to run before using a timer pause - suggestion: use a number between 10 and 10000, but if iteration counters will appear anywhere on screen don't use a round number (i.e. 1000)</param> ///<param name="Delay">Milliseconds to set the timer pause for between stages. Recommended value: 1 - this gives the browser ample time to do "stuff" but avoids unnecassary delay. N.B. the actual delay could be substantially longer than the one specified here due to the way browser events are handled</param> ///<param name="Complete">Function to run when the loop has exited</param> if (IterationsBetweenPauses.IterationsBetweenPauses) { Run = IterationsBetweenPauses.Run; Delay = IterationsBetweenPauses.Delay; Complete = IterationsBetweenPauses.Complete; AtLeastOneDelay = IterationsBetweenPauses.AtLeastOneDelay; Test = IterationsBetweenPauses.Test; IterationsBetweenPauses = IterationsBetweenPauses.IterationsBetweenPauses; } if (AtLeastOneDelay) { setTimeout(StagedLoop, Delay, { Test: Test, Run: Run, IterationsBetweenPauses: IterationsBetweenPauses, Delay: Delay, Complete: Complete, AtLeastOneDelay: false }); return; } var RemainingIterations = IterationsBetweenPauses; while (RemainingIterations > 0) { if (Test && !Test()) { Complete(); return; } Run(); RemainingIterations--; } setTimeout(StagedLoop, Delay, { Test: Test, Run: Run, IterationsBetweenPauses: IterationsBetweenPauses, Delay: Delay, AtLeastOneDelay: AtLeastOneDelay, Complete: Complete }); } And here is an example usage: var I = 0; StagedLoop(4567, 1, false,function () { return I < 10000; }, function () { DoSomethingWith(I); I++; //don't forget to increment! },function(){ //run the next bit of code here }); //N.B. code places here will be run *before* the loop exists (possibly before it runs even a single iteration) //Don't put any code here! This would replace a simple for loop such as this one: for (var I=0;I<10000;I++) { DoSomethingWith(I); } //run the next bit of code here
ok - so it's not quite as clean, but it gets the job done with minimal fuss! Complete test harness below showing the difference in times taken <div id="here">Please Wait... - there is a good chance the browser will "hang" at this point</div> <script type="text/javascript"> setTimeout(RunTest, 100); function RunTest() { var Iterations = 100000; var I; var Start = new Date().getTime(); for (I = 0; I < Iterations; I++) { document.getElementById('here').innerHTML = I; } var End = new Date().getTime(); var Normal = (End - Start); I = 0; Start = new Date().getTime(); StagedLoop(4567, 1, false,function () { return I < Iterations; }, function () { document.getElementById('here').innerHTML = I; I++; }, function () { document.getElementById('here').innerHTML = 'Please wait...'; End = new Date().getTime(); var Staged = (End - Start); console.log('Duration for staged loop: ' + Staged); console.log('Duration for "normal" loop: ' + Normal); document.getElementById('here').innerHTML = 'Duration for "normal" loop: ' + Normal + '<br/>' + 'Duration for staged loop: ' + Staged + '<br/>' + '% performance loss: ' + Math.round(((100.0 * (Staged - Normal)) / Normal)) + '%'; console.log(Staged - Normal); console.log(100.0 * (Staged - Normal)); console.log(((100.0 * (Staged - Normal)) / Normal)); //end inline function }); //N.B. code places here will be run *before* the loop exits (possibly before it runs even a single iteration) //Don't put any code here! } function StagedLoop(IterationsBetweenPauses, Delay, AtLeastOneDelay, Test, Run, Complete) {/// <summary>Stages a loop, inserting timer pauses every n Iterations ///Warning - this method is asyncronus and will exit before looping is completed! /// put code to be run only after looping has finished into the Complete inline function parameter ///</summary> ///<param name="Test">A method to run which will perform the "test" part of the loop to see if looping should continue. This test will be run at the start of the loop (i.e. the loop may run 0 or more times)</param> ///<param name="Run">A method which will be run on every iteration of the loop - don't forget to increment any counters!</param> ///<param name="IterationsBetweenPauses">How many loops to run before using a timer pause - suggestion: use a number between 10 and 10000, but if iteration counters will appear anywhere on screen don't use a round number (i.e. 1000)</param> ///<param name="Delay">Milliseconds to set the timer pause for between stages. Recommended value: 1 - this gives the browser ample time to do "stuff" but avoids unnecassary delay. N.B. the actual delay could be substantially longer than the one specified here due to the way browser events are handled</param> ///<param name="Complete">Function to run when the loop has exited</param> if (IterationsBetweenPauses.IterationsBetweenPauses) { Run = IterationsBetweenPauses.Run; Delay = IterationsBetweenPauses.Delay; Complete = IterationsBetweenPauses.Complete; AtLeastOneDelay = IterationsBetweenPauses.AtLeastOneDelay; Test = IterationsBetweenPauses.Test; IterationsBetweenPauses = IterationsBetweenPauses.IterationsBetweenPauses; } if (AtLeastOneDelay) { setTimeout(StagedLoop, Delay, { Test: Test, Run: Run, IterationsBetweenPauses: IterationsBetweenPauses, Delay: Delay, Complete: Complete, AtLeastOneDelay: false }); return; } var RemainingIterations = IterationsBetweenPauses; while (RemainingIterations > 0) { if (Test && !Test()) { Complete(); return; } Run(); RemainingIterations--; } setTimeout(StagedLoop, Delay, { Test: Test, Run: Run, IterationsBetweenPauses: IterationsBetweenPauses, Delay: Delay, AtLeastOneDelay: AtLeastOneDelay, Complete: Complete }); } </script>

Solving long JSON return from a webservice

I was being plagued by this error intermittently:
Error during serialization or deserialization using the JSON JavaScriptSerializer. The length of the string exceeds the value set on the maxJsonLength property

Some Googling led me to realise that there is indeed a limit, and this limit is quite low.
Most of the posts explained how to increase the limit but also explained why this was a bad idea and that you should instead reduce your JSON size.

I agree with this, better to return what you can to the client and tell them to "try again" for whatever is left.
But this poses a problem - how do you know how much data you can return at once?
Google drew a blank on this one.

This little function here will do the trick:

using System.Configuration; using System.Web.Configuration; public int GetMaxJSONLength() { Configuration cConfig = WebConfigurationManager.OpenWebConfiguration(null); return ((ScriptingWebServicesSectionGroup) cConfig. SectionGroups["system.web.extensions"]. SectionGroups["scripting"]. SectionGroups["webServices"] ).JsonSerialization.MaxJsonLength; }
This can be executed from anywhere, it doesn't have to be part of the web project itself.

Something else to be aware of - any strings in your return value will be encoded.
That means characters like < and > will become \u003c and \u003e - 6 times longer than they were previously.
So if you have a string containing html this can make a substantial difference to the length of the string.

To account for this make sure you use the length of the encoded string, not the original.