DTrace Probing 0.2

What a way to cut it down to the wire.

After about 2 weeks of finishing my job at the Country Club, and with 1 week left in the project. I get this email from Vladimir Vukicevic from Mozilla.

Whoops, just now save this mail when dave pointed it out. What I can
offer as an immediate take is a patch that jonas and I put together for
basically manual probing of a few things. It's not complete, but it'll
be a start.

http://people.mozilla.com/~vladimir/misc/debug-appshell.patch

Basically look at all the printfs; they time certain things, and we'd
probably want probes at the start and end (basically where the timings
start and end). This covers the main chunks of reflow, parsing, and
event handling.

I'd like to see proposed names for all the probes and their location;
I'd suggest xxxxx.start and xxxxx.end. For example, in nsThread.cpp,
the patch has:

diff --git a/xpcom/threads/nsThread.cpp b/xpcom/threads/nsThread.cpp
--- a/xpcom/threads/nsThread.cpp
+++ b/xpcom/threads/nsThread.cpp
@@ -506,9 +506,27 @@ nsThread::ProcessNextEvent(PRBool mayWai

if (event) {
LOG(("THRD(%p) running [%p]\n", this, event.get()));
+
+ PRBool isMainThread = NS_IsMainThread();
+ if (isMainThread)
+ printf ("{");
+
+ PRIntervalTime t0 = PR_IntervalNow();
+
++mRunningEvent;
event->Run();
--mRunningEvent;
+
+ PRIntervalTime t1 = PR_IntervalNow();
+
+ PRUint32 td = PR_IntervalToMilliseconds(t1 - t0);
+ if (isMainThread) {
+ if (td > 100) {
+ printf ("-E(0x%08x,%d)", (unsigned int) (void*) event.get(), td);
+ }
+ printf ("}");
+ }
+
} else if (mayWait) {
NS_ASSERTION(ShuttingDown(), "This should only happen when
shutting down");
rv = NS_ERROR_UNEXPECTED;


So we should have an nsthread event processing start probe at the t0 =
PR_IntervalNow(); for args, initially let's pass down the thread ID and
whether the thread is the main thread or not (or actually, let's just
pass 0 for the thread id if it's the main thread). A second nsthread
event processing end probe should happen at the t1 = PR_IntervalNow bit.

The patch has a bunch of if (t > 100) { ... }, which is just for sanity
sake when printing output -- the probes should not be conditional on how
long things take.

Args for the various probes should just be kept simple for now, we can
figure out later what data we want to pass down.

The big piece that's missing here is painting; since I'm assuming we're
MacOS X only for now, the right place would be in nsChildView.mm in
drawRect -- basically at the start of drawRect and at the end of
drawRect. (I guess a probe isn't really needed, since it's just one
function, but we should have one anyway.)

Getting this stuff in would be a huge help, thanks for taking this on!
I'd love to ship with the dtrace probes enabled in release builds at
some point.

- Vlad

He has been working on this probing portion for Sayrer for some time. And has created a patch to help with debugging the code base. And basically, he would like me to add start and end probe calls on places where

PRIntervalTime xx = PRIntervalNow( );

The first one gets the start, the next one gets the end.

I think I am able to do this, but with 5 days left and the assignment due at 11:59:59 on Saturday - I will really need to cut it down to the wire.

Thankfully, I am gathering all the resources I need to work on it today. And reviewing the patch to make sure I know where things are needed to go.

Hopefully, the patch will compile with and without the probes.

Adding D-Trace to Code Base 0.1 Release

For those who need a reminder of what I am doing: My Project Page

This post recognizes that I have released my 0.1 to the public. I have released:

• Mac OS X 10.5 (Leopard) and OpenSolaris only
  
• Non-extension based
  
• Instructions to compile Mozilla for D-Tracing
  
• Test script to interact with existing probes

Basic Instructions to compiling this are as follows (more advanced and well-explained instructions are on my 0.1 Release - link is at the bottom of the page:

1. Download the Mozilla source tree. The version must be >= 3.0.3
   
2. Extract the tarball at your home directory.
   
3. Within your .mozconfig file. Add this line: ac_add_options --enable-dtrace along with your other options
   
4. Make sure your $PATH variable has at least:
   export PATH=/usr/include/sys:/usr/include:/opt/local/bin:/opt/local/sbin:$PATH
   
5. Compile the source tree.
   
6. Go to your proper executable and run firefox-bin (mozilla/dist/bin is OpenSolaris, mozilla/dist/Minefield.app/Contents/MacOS/
   
7. On a second terminal, create a file called js_callcount.d and add this code to it (use a 2nd terminal than the one handling Minefield):
   

   
   #pragma D option quiet
   
   dtrace:::BEGIN
   {
          printf("Tracing... Hit Ctrl-C to end.\n");
   }
   
   javascript*:::function-entry
   {
          @funcs[basename(copyinstr(arg0)), copyinstr(arg2)] = count();
   }
   
   dtrace:::END
   {
          printf(" %-32s %-36s %8s\n", "FILE", "FUNC", "CALLS");
          printa(" %-32s %-36s %@8d\n", @funcs);
   }
   

   
   
8. Compile and run this script with the second terminal using:
   [sudo] dtrace -s js_callcount.d
   
9. Perform some actions like go to a different web page, hit some buttons, do a Google search etc. without closing Minefield.
   
10. When you think you are done, go to the 2nd terminal and kill the script using Ctrl+C. The output will tell you how many functions were called in each file the D-Trace script picked up on.
    

You may optinally use the command

[sudo] dtrace -n 'javascript*' -l

to display any currently active probes. This will only work while Mozilla is running.

I did a test run of it myself, doing the following scenario:

1. Ran Minefield on first terminal. Created a profile called DTrace and used it
   
2. Ran my D-Script on my second terminal.
   
3. On Minefield - typed [http://www.wikipedia.org/wiki/Santino_Marella http://www.wikipedia.org/wiki/Santino_Marella] and hit Enter.
   
4. Clicked on the first instance of RAW (should be in basic description field on top of wiki).
   
5. Ctrl+C'd out of my D-Script on second terminal.
   
6. Closed Minefield and Ctrl+C'd out of my first terminal.
   

And got these results:

//DTrace::BEGIN
Tracing... Hit Ctrl-C to end.

//Started picking up on my Minefield interactions

//DTrace::END
 FILE                             FUNC                                    CALLS
 Santino_Marella                  writeln                                     2
 WWE_Raw                          writeln                                     2
 autocomplete.xml                 QueryInterface                              2
 autocomplete.xml                 attachController                            2
 autocomplete.xml                 getBoundingClientRect                       2
 autocomplete.xml                 getComplexValue                             2
 autocomplete.xml                 getIntPref                                  2
 autocomplete.xml                 handleEnter                                 2
 autocomplete.xml                 setConsumeRollupEvent                       2
 autocomplete.xml                 stopSearch                                  2
..continues on
autocomplete.xml                 hasChildNodes                              334
 autocomplete.xml                 substr                                    338
 autocomplete.xml                 setAttribute                              344
 autocomplete.xml                 parseInt                                  508
 autocomplete.xml                 min                                       592
 autocomplete.xml                 push                                      610
 javascript&smaxage=21600&maxage=86400 substr                               712
 autocomplete.xml                 getAttribute                             1506
 autocomplete.xml                 charCodeAt                               3876

The probe that handles function-entry (basically when a function is called) seems to handle more on browser.js (was cut off, but appeared) and the xml files than the other locations. A person would need this information to make sure that thier functions are being called the right amount of times. Which helps if a user needs to write more functions, cut off ones that are least used, or access more that can be used to make sure they are "earning thier keep".

To improve on 0.1 so that I can work with 0.2, I only have one thing. There was a patch released on Bugzilla Bug 370906 in Post 20/23 that renames the Probe conventions from javascript* to trace_mozilla*. This is helpful if you have 2 programs running that use the same naming conventions, and maybe the same ID (the * is a wildcard). I was able to run it on Solaris, but not on Mac. I didn't post the instructions that way because I wanted to make sure that I was cross-platform compliant. I want to try to figure out how to get it running on Mac. But I fear that Sayrer (the man who suggested that this technology would be useful for Mozilla) may not like it, or he knows how to solve my problem.

For 0.2, I plan on actually writing custom probes for other sections of the Mozilla Code base. Hopefully, where Sayrer wants them to go. Then I am gonna remodify the above script to check for these "custom probes".

I have written better instructions and easier to copy code on this 0.1 Release wiki. I apologize if my pre tags cut off the code on the screen. It is the way this page is rendered at 1024x768 that bugs me.

Open Source - Done - Now for D-Trace

After much tedious discussions on IRC (between ted and humph), a power outage in the ORI lab and a little GO bus nap, I have got my Mac compiling to work with D-Trace. And I have accomplish 40% of my 0.1 release by doing this.

It turns out, when I compiled it, there is a difference between running the Firefox exe inside the dist, then there is actually running Minefield a little deeper. The reason this was a problem was because Firefox wasn't accepting keyboard input at all (and other programs were).

Now I have D-Trace to look into. From there I can find a probe and learn to use it.

Luckily, I have the research material printed out from the web. And all this will accomplish is make life easier when I actually sit down on the Mac and continue with my 0.1 release.

For those who need a reminder: I am taking D-Trace probes, and adding them to Mozilla Firefox functions. For 0.1, I am taking an existing probe, and using it on one section of the code, then post the results of it to the people. I won't be writing any probes until 0.2.

I had to be able to compile Mozilla on a Mac (accomplished), look up D-Trace and how it works (doing this weekend), then finding an existing probe (weekend), and utilizing it with functions inside the Mozilla code that have a relation (the week).

I have complete confidence that I should be able to get this all done by Saturday. And this assignment is more about time management now than it is actual "grunt work".

Open Source - Compiling Mozilla on Mac

I remember back when I was in Grade 3. Most of our computers at school were old MACs. And I mean the black and white ones with no CD Drives, with no funky colored monitors - and iPods were a vision of the future.

Back then, I wasn't a computer person like I am now. But now, I began to change....

I started using Windows 98 & XP. Good times. Too bad Service Pack 2 on XP wasn't friendly at all!

And Linux became easy to use for me. I had some trouble with vi at first, but then I came to love it like a brother. Then, it happened....

In order for me to begin working with D-Trace, I needed to use OpenSolaris. Problem: I needed a Mac. And it literally took me 20 minutes and a Google search on Safari just to find the location of the Command Prompt :(.

All I had to do in Windows was Windows Key + R > cmd. Linux I just simply opened it on the front screen. But this is Leopard 10.5 :(.

I got adjusted pretty decently to the atmosphere known as Mac OS. But the compilation instructions on Firefox were extremely fuzzy. And I mean EXTREMELY fuzzy. Especially since I didn't know what XTools or Darwinports were.

So I did the next best thing: Went on IRC (nick Hellwolf36) - and had a conversation between ted and jboston. Ted was helpful - his instructions were useful I wanted to get it on a home machine or remote terminal. But I am working in the Open Source lab.

jboston knew what the problem was - it was my PATH! I had already installed GLib and idlLib, XTools, D-Trace on this machine. I just didn't have my PATH set to find it. I pretty much facepalmed and was about to smash my head on the desk (metaphorically speaking).

Other than that, Mozilla Firefox seems to compile and work correctly. I will now share with you people how to avoid this trouble some fate. First, follow these instructions to the letter, especially the Software Requirements part.

Second, you may need to do this to your PATH and possibly your MANPATH.

Finally, you better find something to do for 30-50 minutes ;). You don't exactly want to sit there and watch your build happen.

My next phase for my 0.1 is to learn D-Trace and get a probe to work for me. One week left is the real nail-biter. But I have discovered, over the many years I work - that sometimes pressure brings out the best in all of us.

Open Source - Compiling Mozilla with D-Trace

I begin my step up to 0.1 glory with the bread and butter - compiling Mozilla with D-Trace. 

It's quite simple: Simply add ac_add_options --enable-dtrace in the .mozconfig file. And pray that nothing will explode.

But now, what am I to do in a lab for 1 hour?

If I was at home, I would probably log onto SCHTHACK and play Phantasy Star Online. Or boot up ZSNES for some Secret of Mana II (Seiken Densetsu 3). But considering I am on a tight schedule for 0.1 (which is due Saturday), and I don't want to let David Humphrey's links on D-Trace go to waste - I will be learning technology that is foriegn to me.

This will be by 1st time on a Mac. And I will openly admit, I am not a Mac person, but I can use Linux - so that has some merit. As long as I know what D-Trace is and how it works, I should be ready to get some things going. I wouldn't be surprised if I got half of this assignment done in 1 day. But it should be exciting, if I manage to pull it off.

Well, hopefully everything works out for the better. And given the fact this course is more time-based than difficulty - Failure is not an option.

Open Source - Time to Start

The time has come to actually do something useful for this course. WOW! Useful!!!

For the first month, I have been basically either working, playing a little PSO here and there, or forced to do other assignments - such as my Game Modeling course.

To help us in the right direction, Professor Humphrey shows us Bugzilla. It is a website that allows people who are working with Mozilla's technology to post bugs, to allow for fixes, or for people to work on them. They can then post thier findings here.

He asks us to take 3 bugs found on this page, and add them to our "Project Wiki Page". Then follow a person to see what kind of findings they create.

It looked simple for some people - cause a vast majority of people who are in my course either got bug fixing, testing, or updating something that exist. But alas, I don't.

I am basically creating probe programs via D-Trace to help Mozilla debug using that software. But this is something a whole lot of people haven't done before. All the sources I have are from my "mystery contact" in IRC (we crash at the freenode or moznet servers @ #seneca). I need to meet him this week after I finish a probing program in C++.

All and all, Bugzilla was fruitless for me. But it isn't all that bad. What if there is a bug I see, and my "probing software" finds the problem for the user. That would save MANY people a lot of time, and a lot of headbangings on the table or wall :).

Open Source - Diving into Source Code

Hooray for more adventures!

Last week, it was learning how to install this source code. This week, it is about actually seeing the components that make Mozilla Firefox, well... Firefox.

David Humphrey (my professor in the OSD course) offered us a "code search engine"

It basically allows you to "Google" things inside Mozilla code for certian functions, definitions, etc.

I decided to search how Mozilla uses Bookmark Page to keep track of your favorite pages. So I began searching the option itself. Which lead me to a UI definition file, and had the name of the function. That was simple, but it needed more.

Things got complicated there, I was basically searching through 7 files, until I found out that BookmarkPage (the name of a function) was an alias to a Javascript function.

And to another surprise the function PCH_bookmarkPage was only about 60 lines long, excluding closing brackets being on seperate lines (I often do that myself).  Not a bad length for saving your webpages.

The function itself basically stores any information about your page in Javascript variables, and then sends you a prompt asking if you want to Bookmark the page. Clicking yes proceeds on to adding your page to the Bookmarks section, no is for ignoring it.

But alas, the "fun" is just beginning. I must now think of a plan for my 0.1 release of the D-Trace program for Mozilla. Because of working at this golf club...

things got out of hand. But in life, we all need to make sacrifices to do something else, just some more than others. The life of a future programmer :(.