SAS Macro to Make Tiny URLs

Someone on SAS-L wanted a piece of SAS code to convert a long url to a short one. Well, here you go:

filename in "x:\temp\in";
filename out "x:\temp\out.txt";

%macro MakeTiny(longUrl=);
 data _null_;
    file in lrecl=1028;
    put "url=&longUrl" ;
    run;

    proc http in=in out=out url="http://tinyurl.com/api-create.php"
      method="post"
      ct="application/x-www-form-urlencoded";
 run;

 data _null_ ;
    infile out;
    input tinyUrl :$1024. ;
    call symput('tinyUrl',tinyUrl);
    %global tinyUrl ;
    run;

%mend makeTiny;

%MakeTiny(longUrl=www.savian.net);

%put &tinyUrl ;

WCF and PROC SOAP

Adam Bullock in SAS Tech Support has become a superstar in my book. I don't send him tickets directly but the area I am working in always seems to find him.

The latest issue was consuming a WCF service. This was different since Microsoft uses interfaces in WCF vs the old way of doing it. I didn't catch that but Adam did. Here is working SAS code for handling it:

FILENAME REQUEST 'C:\temp\REQUEST.xml';
FILENAME RESPONSE 'C:\temp\RESPONSE.xml';

data _null_;
file request;
input;
put _infile_;
datalines4;
<soapenv:Envelope xmlns:soapenv="http://schemas.xmlsoap.org/soap/envelope/" xmlns:tem="http://tempuri.org/">
   <soapenv:Header/>
   <soapenv:Body>
      <tem:DoWork>
         <!--Optional:-->
         <tem:p>Test</tem:p>
      </tem:DoWork>
   </soapenv:Body>
</soapenv:Envelope>
;;;;
run;

%let RESPONSE=RESPONSE;

proc soap in=REQUEST
out=&RESPONSE
url="http://prognos2.savian.net/SampleServices/cdmAlive.svc"
soapaction="http://tempuri.org/ICdmAlive/DoWork"
;
run;

Adam used SoapUI to track it down and was nice enough to send a picture of where he saw the interface call:

Wix and InStyler

This isn't a SAS post even though SAS is on the periphery of this one. This post is designed to help other developers in a similar boat if they get a hit on the error message verbiage.

The standard MS Installer is going away (next year, I believe), and a lot of people are converting to WIX (Windows Installer XML). On my latest project, I needed a custom installer. Now anyone who has ever worked with custom installers should be able to tell you what an absolute pain it is, how hard it is to debug, hard to put in custom screens, etc.

This seemed like an opportune time to jump over to WIX, especially when I needed a lot more than what the custom dialogs could provide under the standard Windows installer technology inside of Visual Studios. InstallShield was NOT an option. They want way, way too much money for their product and I am not a fan from days of yore.

WIX is very flexible but it is also hard to work with. There are no GUIs, per se, for it which means a lot of manual coding. My current WIX file, code generated, is weighing in at 900+ lines of XML. That and looking up the GUIDs for products, etc. meant a lot of manual effort and lots of places for mistakes.

I found a product on the web called Instyler Setup. Not sure how well supported it is, and it has a number of bugs, but it mostly writes the WIX for you. Two of the most annoying bugs I wanted to describe so others can seee them on the web:

1. "Error 2834: The next pointers on the dialog ErrorPopup do not form a single loop"

...when running the installer is caused by the text label on the ErrorPopup dialog having a tabstop set to true. MSI is very, very picky about everything which is why it is a nightmare to debug.

2. "Error 1723. There is a problem with this Windows Installer package."

...make sure you reference the CA dll instead of teh normal dll on your custom action.

3. To make working with WIX a lot easier, download and install the DTF package:

http://wix.sourceforge.net/

4. Turn on MSI debugging in your local group policy. this makes like a lot easier to track things down.

I hope these get picked up in the search engines and helps some other dev out at some point.

IIS 7.5 and SAS DCOM: ACCESS DENIED, ACCESS DENIED, ACCESS DENIED

And the joy of being a pathfinder continues ;-]

IIS 7.5 has now changed the game. No longer do we play with NETWORK SERVICES to get SAS operational. Now it is a new user IIS APPPOOL\DefaultAppPool. Security has now been taken from NETWORK SERVICES, which covered the website, to specific security by the AppPools. I am not a security expert but that is my understanding.

So I took my happy little web services project (WCF) that worked fine in Visual Studio 2010 (Cassini), built a Web Setup for it, and deployed it to localhost. Then spent the next 7 hours staring at the same error: ACCESS DENIED. This was occurring on CreateObjectByServer. ACCESS DENIED, ACCESS DENIED, ACCESS DENIED, over and over again. I spent so much time in DCOMCNFG even going so far as to enable the Administrator account as the launching account, open up everything I could, and still ACCESS DENIED.

This morning I spoke to Bubba in SAS Tech Support. Great guy. He didn’t know this area but diligently was tossing out terms on tech notes he saw. He finally tossed out “enable 32-bit apps” and I keyed in on that one immediately. Well, I did a bit of investigation and have now made it far enough along for a blog post:

In IIS 7.5, get your application in place under a Virtual Directory:



Set up whatever app pool you are using. In this case, it is DefaultAppPool. Click on the Application Pools under the directory. Now here is where you have a choice. You can use the DefaultAppPool for everything but that also means giving it access to every library location, file, whatever. I opted to instead set up a new account called ‘WebService”.

Now it needs to be associated with the access rights for each directory and file. This is standard Windows stuff. Click on a directory and add the new username:




In IIS, change the App Pool to use the new identity:




Here are the settings and the ones to be concerned with:




Enable 32-bit applications and set the identity as needed. By default, it will be the name of your app pool. We will use our newly created name of WebService.

Now, time to go into DCOMCNFG (Start --> Run --> DCOMCNFG and fix the DCOM entry for the SAS Workspace:




Finally, change your code to the following:

                ObjectFactory factory = new ObjectFactory();
                ServerDef server = new ServerDef();
                SasWorkSpace = (Workspace)factory.CreateObjectByServer("ws", true, server, "WebService", "MyPassword");
                var id = SasWorkSpace.UniqueIdentifier;
                SasLanguageService = SasWorkSpace.LanguageService;
                keeper = new ObjectKeeper();
                keeper.AddObject(1, "SASServer", SasWorkSpace);

No more ACCESS DENIED.

Thanks Bubba. I think we p*wned it.

…not really but it works good enough for me for now ;-)

SAS StoredProcessService events

If you encounter the following error when using SAS's StoredProcessService events:

event invocation for COM objects requires event to be attributed with DispIdAttribute

Roll your project back from .NET 4.0 to .NET 3.5. This may only be applicable to the web services code.

Example code:

class Program
{
static void Main(string[] args)
{

Workspace ws = new Workspace();
SAS.LanguageService ls = ws.LanguageService;
ls.StepError += new CILanguageEvents_StepErrorEventHandler(ls_StepError); <---- ERROR OCCURS HERE
ls.Submit("data test; abort; run;");
}
static void ls_StepError()
{
Console.WriteLine("Bingo!");
Console.ReadLine();
}
}

Flush with excitement...errr, logs

Well, it is Friday night, I am back from the doctor with antibiotics due to a possible spider bite (love those little guys), and I decide to track down some performance issues with the SAS StoredProcessService (or Microsoft's Cassini). Somewhere, someone is responsible. Set up my little test bed to see it all work:


SAS.Workspace ws = new Workspace();
LanguageService ls = ws.LanguageService;
StoredProcessService sp = ls.StoredProcessService;
sp.Repository = @"file:" + @"x:\temp";
sp.Execute("test.sas", string.Empty);


All is well and good. Life is happy, kids are playing, sunshine is everywhere.

Wait! We need to see if the code ran. let's check the log:


string log = ls.FlushLog(1000);

AHHHH!!!! It never comes back. Where is my string!?!?

Turns out that if the FlushLog int amount EXCEEDS the total number of chars in the log, you go to a happy place called infinity. Less than the total chars and all is well.

I decided to test this a few times and was able to get a picture of my CPU utilization:















This does not happen when using the LanguageService directly, only when using the StoredProcessService. Time to wrap up this blog, pull back from the black hole of infinite loops, and go issue a bug report with SAS R&D.

If any SAS R&D folks in this area read this blog, let me know if I missed something. Right now, it seems like I have been bitten twice today by bugs.

3-fer on StoredProcessService

Ok, well the major issues have been tackled except 1. How do you handle spaces in your NameValuePairs? Well, without the single quotes below, it kept splitting the values between the Sample and the Data. The single quotes hold it together.

Check this out:


string[] parms = new string[2]
{
"outdata=WORK.ALAN_20110418_070053",
@"datalib='X:\Data\Prognos\DemandForecasting\Sample Data'",
};


Notice the single quotes around the value? It took a long time to track that one down (thanks ThotWave).

Now it is simple to submit to SAS:


string newParms = string.Empty;
if (parms != null)
{
newParms = String.Join(" ", parms);
}
StoredProcessService sp = Common.SasLanguageService.StoredProcessService;
sp.Repository = storedProcLibrary;
sp.Execute(storedProcedureName, newParms);


I would ask R&D a simple question which is why isn't there an option on the SPS to specify the delimiter or am I missing a flag somewhere?

Our journey with SAS's StoredProcessService object continues...

Check out this code:


string newParms = "outdata=WORK.TEST";
Common.SasLanguageService.Async = true;
StoredProcessService sp = Common.SasLanguageService.StoredProcessService;
sp.Repository = storedProcLibrary;
sp.Execute(storedProcedureName, newParms);

Standard stuff when working with the stored process server through VB or C# (or any other means of hitting these dlls).

Funny thing is the SAS log shows that the macro assignment of newParms never takes place. Hence, &outdata is undefined:

SYMBOLGEN: Macro variable OUTDATA resolves to

Commenting out the Async makes this all work:

SYMBOLGEN: Macro variable OUTDATA resolves to WORK.ALAN_20110418_070053

Now, even wiring up the event handlers for SubmitComplete do not fix the issue. Seems like a bug but I'll let the guys in R&D figure it out. If you have to waste a lot of time on it, however, keep in mind the nuances here until SAS R&D dives in.

Well, back out to the far left field where I hang out. Time to work on the business problem.

This one is a real Keeper

So, if you ever get this error while trying to work with OleDb and SAS IOM:

The object ... could not be found; make sure it was previously added to the object keeper

Make sure that you include the following lines (the lack of a Keeper is what causes this error):

ObjectFactory factory = new ObjectFactory();
ServerDef server = new ServerDef();
Workspace ws = (Workspace)factory.CreateObjectByServer("ws", true, server, "", "");
ObjectKeeper keeper = new ObjectKeeper();
keeper.AddObject(1, "SASServer", ws);

That whole ObjectKeeper is missing from a number of examples that I have seen and it is critical for everything to function. It becomes apparent when trying to get the data from an OleDb DataAdaptor.

Wufoo and REST API

Wufoo (http://www.wufoo.com/) is a great service for creating user forms and getting input. Build a form interactively, let people fill it out, and Wufoo collects the results. If you haven't checked it out before, I highly recommend this great service.

I was simply parsing the emails when they were sent upon form completion but that is a real hassle due to HTML. However, Wufoo has a really good REST API so I started playing around with it. Here, in C#, is how I used their API to get my data. This can easily be converted into SAS as well:

The HttpGet:

internal static string HttpGet(string uri, string apiKey)
{
    var pairs = new StringBuilder();
    var req = WebRequest.Create(uri) as HttpWebRequest;
    req.Timeout = 300000;
    req.Credentials = new NetworkCredential(apiKey, string.Empty);
    req.UserAgent = "AlanC";
    req.ContentType = "text/html";
    req.Method = "GET";

    // Get response

    using (var response = req.GetResponse() as HttpWebResponse)
    {
        if (response != null)
        {
            WebResponse resp = req.GetResponse();
            if (resp != null)
            {
                var sr = new StreamReader(resp.GetResponseStream());
                return sr.ReadToEnd().Trim();
            }
            return null;
        }
    }
    return null;
}

The call to the method:

XDocument xd = XDocument.Parse(HttpGet(@"https://xxxxxxx.wufoo.com/api/v3/forms/xxxxx3/entries.xml", "XXXX-XXXX-XXXX-XXXX"));

The https://xxxxx would correspond to your domain on wufoo.
The xxxxx after forms in the uri would be the form id assigned by Wufoo.
The XXX-XXXX-etc. is the API Key assigned by Wufoo.

The above can be found on the Wufoo site. Just look at the Wufoo API information to find out how to obtain it.

SAS Transport Files and .NET

Well, someone contacted me about the Data Management Utilities. they loved them (thank you) but wanted to know about reading SAS Transport Files. Well, after fiddling with the localProvider a bit and getting nowhere, I stumbled onto something cool:

• Download and install the SAS Universal Viewer from the SAS support site
• Create a new project in Visual Studio
• Add a reference from the SAS Universal Viewer install files to the following 2 dlls
•    SAS.UV.Transport
•    SAS.UV.Utility
• Use the following C# code (adapt as needed):

TransportFile tf = new TransportFile(@"x:\temp\sample.xpt");
var x = tf.Datasets;

Your dataset will be in variable x

UPDATED: 11/8/2017

[NOTE: I have also created a direct reader for the XPT binary format. That took a long time (a week, maybe). Contact me if you need that]

Complete code:

using System;
using System.Collections.Generic;
using System.Data;
using System.Data.OleDb;
using System.Linq;
using System.Text;
using System.Threading.Tasks;
using SAS.UV.Transport;

namespace ReadSasXportFiles
{
    class Program
    {

        static void Main(string[] args)
        {
            var tf = new TransportFile(@"x:\temp\shoes.xpt");
            var x = tf.Datasets;

         }

     }

}

IMPORTANT NOTE: Make sure your build is set for x64 and not Any CPU. You will see a warning:

Warning There was a mismatch between the processor architecture of the project being built "MSIL" and the processor architecture of the reference "SAS.UV.Transport", "AMD64".

RegexBuddy and Automated Emails

So, I am going to help out with a computer club at my kids middle school. Since a topic was needed to start it all off, I said 'hey! regular expressions are used everywhere, why not start there. Plus, it is very useful.'.

So, I sent JGSoft (the owner of RegexBuddy) an email and explained the situation. How they were kids, we needed a limited version or a free one for a certain time, the kids may purchase it after the class, etc.

What was the response?

"We indeed do not offer a free trial version of RegexBuddy for download on our web site. At RegexBuddy's low price point and with our solid 3-month money-back guarantee, you can buy the full version of RegexBuddy entirely risk-free."

All of the supportive messages for RegexBuddy on SAS-L and other places and the best they can do is some automated response saying all of the kids need to cough up the 40 dollars to use their product. This is a fine way to get a bad reputation IMO.

Now I need to go get a free regex editor to show the kids despite RegexBuddy being the premiere product on the market. I am trying to turn them into kids interested in the product so they can go home and dazzle mom and dad who will then pay the fee.

Ok, off of my soapbox.