DTrace Probing 0.2

What a way to cut it down to the wire.

After about 2 weeks of finishing my job at the Country Club, and with 1 week left in the project. I get this email from Vladimir Vukicevic from Mozilla.

Whoops, just now save this mail when dave pointed it out. What I can
offer as an immediate take is a patch that jonas and I put together for
basically manual probing of a few things. It's not complete, but it'll
be a start.

http://people.mozilla.com/~vladimir/misc/debug-appshell.patch

Basically look at all the printfs; they time certain things, and we'd
probably want probes at the start and end (basically where the timings
start and end). This covers the main chunks of reflow, parsing, and
event handling.

I'd like to see proposed names for all the probes and their location;
I'd suggest xxxxx.start and xxxxx.end. For example, in nsThread.cpp,
the patch has:

diff --git a/xpcom/threads/nsThread.cpp b/xpcom/threads/nsThread.cpp
--- a/xpcom/threads/nsThread.cpp
+++ b/xpcom/threads/nsThread.cpp
@@ -506,9 +506,27 @@ nsThread::ProcessNextEvent(PRBool mayWai

if (event) {
LOG(("THRD(%p) running [%p]\n", this, event.get()));
+
+ PRBool isMainThread = NS_IsMainThread();
+ if (isMainThread)
+ printf ("{");
+
+ PRIntervalTime t0 = PR_IntervalNow();
+
++mRunningEvent;
event->Run();
--mRunningEvent;
+
+ PRIntervalTime t1 = PR_IntervalNow();
+
+ PRUint32 td = PR_IntervalToMilliseconds(t1 - t0);
+ if (isMainThread) {
+ if (td > 100) {
+ printf ("-E(0x%08x,%d)", (unsigned int) (void*) event.get(), td);
+ }
+ printf ("}");
+ }
+
} else if (mayWait) {
NS_ASSERTION(ShuttingDown(), "This should only happen when
shutting down");
rv = NS_ERROR_UNEXPECTED;


So we should have an nsthread event processing start probe at the t0 =
PR_IntervalNow(); for args, initially let's pass down the thread ID and
whether the thread is the main thread or not (or actually, let's just
pass 0 for the thread id if it's the main thread). A second nsthread
event processing end probe should happen at the t1 = PR_IntervalNow bit.

The patch has a bunch of if (t > 100) { ... }, which is just for sanity
sake when printing output -- the probes should not be conditional on how
long things take.

Args for the various probes should just be kept simple for now, we can
figure out later what data we want to pass down.

The big piece that's missing here is painting; since I'm assuming we're
MacOS X only for now, the right place would be in nsChildView.mm in
drawRect -- basically at the start of drawRect and at the end of
drawRect. (I guess a probe isn't really needed, since it's just one
function, but we should have one anyway.)

Getting this stuff in would be a huge help, thanks for taking this on!
I'd love to ship with the dtrace probes enabled in release builds at
some point.

- Vlad

He has been working on this probing portion for Sayrer for some time. And has created a patch to help with debugging the code base. And basically, he would like me to add start and end probe calls on places where

PRIntervalTime xx = PRIntervalNow( );

The first one gets the start, the next one gets the end.

I think I am able to do this, but with 5 days left and the assignment due at 11:59:59 on Saturday - I will really need to cut it down to the wire.

Thankfully, I am gathering all the resources I need to work on it today. And reviewing the patch to make sure I know where things are needed to go.

Hopefully, the patch will compile with and without the probes.

Adding D-Trace to Code Base 0.1 Release

For those who need a reminder of what I am doing: My Project Page

This post recognizes that I have released my 0.1 to the public. I have released:

• Mac OS X 10.5 (Leopard) and OpenSolaris only
  
• Non-extension based
  
• Instructions to compile Mozilla for D-Tracing
  
• Test script to interact with existing probes

Basic Instructions to compiling this are as follows (more advanced and well-explained instructions are on my 0.1 Release - link is at the bottom of the page:

1. Download the Mozilla source tree. The version must be >= 3.0.3
   
2. Extract the tarball at your home directory.
   
3. Within your .mozconfig file. Add this line: ac_add_options --enable-dtrace along with your other options
   
4. Make sure your $PATH variable has at least:
   export PATH=/usr/include/sys:/usr/include:/opt/local/bin:/opt/local/sbin:$PATH
   
5. Compile the source tree.
   
6. Go to your proper executable and run firefox-bin (mozilla/dist/bin is OpenSolaris, mozilla/dist/Minefield.app/Contents/MacOS/
   
7. On a second terminal, create a file called js_callcount.d and add this code to it (use a 2nd terminal than the one handling Minefield):
   

   
   #pragma D option quiet
   
   dtrace:::BEGIN
   {
          printf("Tracing... Hit Ctrl-C to end.\n");
   }
   
   javascript*:::function-entry
   {
          @funcs[basename(copyinstr(arg0)), copyinstr(arg2)] = count();
   }
   
   dtrace:::END
   {
          printf(" %-32s %-36s %8s\n", "FILE", "FUNC", "CALLS");
          printa(" %-32s %-36s %@8d\n", @funcs);
   }
   

   
   
8. Compile and run this script with the second terminal using:
   [sudo] dtrace -s js_callcount.d
   
9. Perform some actions like go to a different web page, hit some buttons, do a Google search etc. without closing Minefield.
   
10. When you think you are done, go to the 2nd terminal and kill the script using Ctrl+C. The output will tell you how many functions were called in each file the D-Trace script picked up on.
    

You may optinally use the command

[sudo] dtrace -n 'javascript*' -l

to display any currently active probes. This will only work while Mozilla is running.

I did a test run of it myself, doing the following scenario:

1. Ran Minefield on first terminal. Created a profile called DTrace and used it
   
2. Ran my D-Script on my second terminal.
   
3. On Minefield - typed [http://www.wikipedia.org/wiki/Santino_Marella http://www.wikipedia.org/wiki/Santino_Marella] and hit Enter.
   
4. Clicked on the first instance of RAW (should be in basic description field on top of wiki).
   
5. Ctrl+C'd out of my D-Script on second terminal.
   
6. Closed Minefield and Ctrl+C'd out of my first terminal.
   

And got these results:

//DTrace::BEGIN
Tracing... Hit Ctrl-C to end.

//Started picking up on my Minefield interactions

//DTrace::END
 FILE                             FUNC                                    CALLS
 Santino_Marella                  writeln                                     2
 WWE_Raw                          writeln                                     2
 autocomplete.xml                 QueryInterface                              2
 autocomplete.xml                 attachController                            2
 autocomplete.xml                 getBoundingClientRect                       2
 autocomplete.xml                 getComplexValue                             2
 autocomplete.xml                 getIntPref                                  2
 autocomplete.xml                 handleEnter                                 2
 autocomplete.xml                 setConsumeRollupEvent                       2
 autocomplete.xml                 stopSearch                                  2
..continues on
autocomplete.xml                 hasChildNodes                              334
 autocomplete.xml                 substr                                    338
 autocomplete.xml                 setAttribute                              344
 autocomplete.xml                 parseInt                                  508
 autocomplete.xml                 min                                       592
 autocomplete.xml                 push                                      610
 javascript&smaxage=21600&maxage=86400 substr                               712
 autocomplete.xml                 getAttribute                             1506
 autocomplete.xml                 charCodeAt                               3876

The probe that handles function-entry (basically when a function is called) seems to handle more on browser.js (was cut off, but appeared) and the xml files than the other locations. A person would need this information to make sure that thier functions are being called the right amount of times. Which helps if a user needs to write more functions, cut off ones that are least used, or access more that can be used to make sure they are "earning thier keep".

To improve on 0.1 so that I can work with 0.2, I only have one thing. There was a patch released on Bugzilla Bug 370906 in Post 20/23 that renames the Probe conventions from javascript* to trace_mozilla*. This is helpful if you have 2 programs running that use the same naming conventions, and maybe the same ID (the * is a wildcard). I was able to run it on Solaris, but not on Mac. I didn't post the instructions that way because I wanted to make sure that I was cross-platform compliant. I want to try to figure out how to get it running on Mac. But I fear that Sayrer (the man who suggested that this technology would be useful for Mozilla) may not like it, or he knows how to solve my problem.

For 0.2, I plan on actually writing custom probes for other sections of the Mozilla Code base. Hopefully, where Sayrer wants them to go. Then I am gonna remodify the above script to check for these "custom probes".

I have written better instructions and easier to copy code on this 0.1 Release wiki. I apologize if my pre tags cut off the code on the screen. It is the way this page is rendered at 1024x768 that bugs me.

Open Source - Compiling Mozilla on Mac

I remember back when I was in Grade 3. Most of our computers at school were old MACs. And I mean the black and white ones with no CD Drives, with no funky colored monitors - and iPods were a vision of the future.

Back then, I wasn't a computer person like I am now. But now, I began to change....

I started using Windows 98 & XP. Good times. Too bad Service Pack 2 on XP wasn't friendly at all!

And Linux became easy to use for me. I had some trouble with vi at first, but then I came to love it like a brother. Then, it happened....

In order for me to begin working with D-Trace, I needed to use OpenSolaris. Problem: I needed a Mac. And it literally took me 20 minutes and a Google search on Safari just to find the location of the Command Prompt :(.

All I had to do in Windows was Windows Key + R > cmd. Linux I just simply opened it on the front screen. But this is Leopard 10.5 :(.

I got adjusted pretty decently to the atmosphere known as Mac OS. But the compilation instructions on Firefox were extremely fuzzy. And I mean EXTREMELY fuzzy. Especially since I didn't know what XTools or Darwinports were.

So I did the next best thing: Went on IRC (nick Hellwolf36) - and had a conversation between ted and jboston. Ted was helpful - his instructions were useful I wanted to get it on a home machine or remote terminal. But I am working in the Open Source lab.

jboston knew what the problem was - it was my PATH! I had already installed GLib and idlLib, XTools, D-Trace on this machine. I just didn't have my PATH set to find it. I pretty much facepalmed and was about to smash my head on the desk (metaphorically speaking).

Other than that, Mozilla Firefox seems to compile and work correctly. I will now share with you people how to avoid this trouble some fate. First, follow these instructions to the letter, especially the Software Requirements part.

Second, you may need to do this to your PATH and possibly your MANPATH.

Finally, you better find something to do for 30-50 minutes ;). You don't exactly want to sit there and watch your build happen.

My next phase for my 0.1 is to learn D-Trace and get a probe to work for me. One week left is the real nail-biter. But I have discovered, over the many years I work - that sometimes pressure brings out the best in all of us.

Open Source - Compiling Mozilla with D-Trace

I begin my step up to 0.1 glory with the bread and butter - compiling Mozilla with D-Trace. 

It's quite simple: Simply add ac_add_options --enable-dtrace in the .mozconfig file. And pray that nothing will explode.

But now, what am I to do in a lab for 1 hour?

If I was at home, I would probably log onto SCHTHACK and play Phantasy Star Online. Or boot up ZSNES for some Secret of Mana II (Seiken Densetsu 3). But considering I am on a tight schedule for 0.1 (which is due Saturday), and I don't want to let David Humphrey's links on D-Trace go to waste - I will be learning technology that is foriegn to me.

This will be by 1st time on a Mac. And I will openly admit, I am not a Mac person, but I can use Linux - so that has some merit. As long as I know what D-Trace is and how it works, I should be ready to get some things going. I wouldn't be surprised if I got half of this assignment done in 1 day. But it should be exciting, if I manage to pull it off.

Well, hopefully everything works out for the better. And given the fact this course is more time-based than difficulty - Failure is not an option.

Open Source - Diving into Source Code

Hooray for more adventures!

Last week, it was learning how to install this source code. This week, it is about actually seeing the components that make Mozilla Firefox, well... Firefox.

David Humphrey (my professor in the OSD course) offered us a "code search engine"

It basically allows you to "Google" things inside Mozilla code for certian functions, definitions, etc.

I decided to search how Mozilla uses Bookmark Page to keep track of your favorite pages. So I began searching the option itself. Which lead me to a UI definition file, and had the name of the function. That was simple, but it needed more.

Things got complicated there, I was basically searching through 7 files, until I found out that BookmarkPage (the name of a function) was an alias to a Javascript function.

And to another surprise the function PCH_bookmarkPage was only about 60 lines long, excluding closing brackets being on seperate lines (I often do that myself).  Not a bad length for saving your webpages.

The function itself basically stores any information about your page in Javascript variables, and then sends you a prompt asking if you want to Bookmark the page. Clicking yes proceeds on to adding your page to the Bookmarks section, no is for ignoring it.

But alas, the "fun" is just beginning. I must now think of a plan for my 0.1 release of the D-Trace program for Mozilla. Because of working at this golf club...

things got out of hand. But in life, we all need to make sacrifices to do something else, just some more than others. The life of a future programmer :(.

Open Source - Playing with Ubiquity

Well, it's time yet for another blog post, fortelling more "nightmares" with... Open source.

This time, I got into a little Mozilla Firefox 3.0 addon called Ubiquity. It is a program that allows you to type commands and Mozilla will do things with them regarding web pages and such.

For instance, typing wikipedia LOL inside of its command-line will open a wikipedia page for LOL.

However, it is pretty limited. It can only do certian things at first, but seeing how Firefox is open-source, other poeple starting making Ubiquity commands to make it do interesting things. You can make it search for NASDAQ quotes, search on Mininova.org for certian torrents, or activate a search engine to search for images or such.

I did something relatively simple. Benny Sheerin created an open command. I took it and rewrote parts of it to make a goto command using Javascript (the base language the Ubiquity commands are written in). It works like a Hash Table in Perl programming. There would be a key (an alias or keyword), and then would be a value associated with it (in this case, a webpage). 

When you have so many favorites pages or you dont have the hard drive space to store these pages on your PC, then this little add-on is right for you. All you need to do to add a new page, is create a new key-value pair inside the Javascript program. And then re-add it to the Ubiquity command list.

The biggest problem I ran into was compiling it. I did read over Benny's existing code well, it's just - I personally SUCK at Javascript. I hate the language personally, it was the sole reason my hopes of getting an A in INT222 went down the tube. So I had to perservere :( . JSLint (JS Compiler) wouldn't cooperate on a couple of lines and was giving me odd errors. After sitting in front of the PC for about 3 hours, and a few rounds of PSO, I managed to get it operational

I am definitly interested in this program and it's capabilities. If I write another program for this, I would make it so you can type in a word, and then a forum (running on vBulletin) and it would open the Search results to said forum. The only problem with this: You may need to log onto said forum, so you can use the Search tool. And I would also have to write it so it either shows results as threads, or posts. To start, I would need to find an existing search program, and rewrite it to search on boards instead of search engines.

If you wish to gain access to the Javascript code and use it for yourself, or you wish to "borrow" it and use it for your diabolical and/or productive schemes, you may find it here, along with other scripts written by my fellow comrades.