The power of words in debugging

[R.G.]

The process of debugging a problem in a pedal from a written description is one of matching the words against a previous pile of defects found in other stuff. The main reason that beginners can't do this is that their pile of defects-found, their experience collection, is essentially nil. To a beginner, any defect could have been caused by ANYTHING, including the color of socks they put on.   

Ideally, one would know enough about circuits that one could effectively "simulate" the circuit in one's head, at least parts of it. But this skill takes some time to develop.

But I'm wandering.

Here are some key words I realized I pick up on as being highly, highly indicative and the possible conditions they show. This is another version of the stuff I've written down in the debugging FAQs and guides, and a "prequel" to "what to do when it doesn't work".

When you hear:	think about:
gated	misbiasing
hum	ground wiring issues
hum	rectifiers and filters, if present
hum	is this 50/60Hz or 100/120Hz?
hum	open input wiring
hiss	high gain
hiss	rf oscillation
no sound	is there power getting to it?
no sound	is the signal wiring broken somewhere?
no sound	short, especially solder blobs
drops out	intermittent contact
fades in/out	cap charging up/down
crackles	intermittent contact
intermittent contact	cold/cracked solder joint
intermittent contact	dirty pot, jack, or switch contacts
crackling pot	DC across a signal pot
"... gets hot"	"... too much current"

I recently read that you start out with a bag of luck and an empty bag of experience. Success is filling up your experience bag before you run out of luck.

[digi2t]

The pilot banked on banking his plane over the left bank.

Hum....

[Mark Hammer]

When I was in grad school, I used to follow the research more closely, but the details of relevance haven't changed much in the intervening 18 years.

The difference between "experts" and "novices", in virtually any domain, show up in the manner in which their knowledge is organized.  In a sense, gaining expertise in any area involves not only acquiring knowledge, but organizing it for maximum efficiency.  And when researchers look at experts closely, one of the things they regularly see is that experts are able to very quickly identify what aspects/segments of their knowledge in the problem area do NOT apply to the current problem under consideration.  Quickly eliminating what you don't have to think about helps to speed up the process, and also reduce demands on the thinker.

Troubleshooting pedals is just one more area of expertise, no different than automotive repair, radiology, or responding to mystery baskets on the Iron Chef.  A novice may dutifully abide by the protocol laid put here - http://www.diystompboxes.com/wiki/index.php?title=Debugging - and follow it algorithmically, step-by-step, without any sense of priorities.  That's not "wrong", just tedious and frustrating for many.  As R.G.'s post illustrates, over time, and with experience, one comes to know the shortcuts, and the higher probability pairings of observed malfunctions, likely causes, and most diagnostic indicators (i.e., signs to look for).

For example, things that vary in graded fashion with time are often connected to "states" that vary non-instantaneously with time, like charge-up (i.e., caps) or heat build-up.  Things that do change in instantaneous fashion will often, if not always, have something to do with continuity.  Of course, the locus of the discontinuity is not easy to nail down unless one has additional information, and knowing what sorts of information allow you check off "it's not that" quickly for a bunch of possibilities, is useful.  Indeed, when people post about something that's not working, a lot of the initial questions that the "help crew" ask are as much intended to narrow down what it likely isn't, as much as would it possibly is.

[defaced]

Indeed, when people post about something that's not working, a lot of the initial questions that the "help crew" ask are as much intended to narrow down what it likely isn't, as much as would it possibly is.

Yep.  I almost always ask for three things: schematic, layout, and pics.  I first want to see what is most likely causing/not causing the problem.  A good schematic and layout means it's probably a build issue.  A bad schematic means it's a design problem. And a bad layout means someone probably didn't think of the stray R, C and L of the board layout, or they used something that doesn't link the schematic and the layout which introduces the beloved human factor.  After that, then I start to work toward details like voltages, signal tracing and such.  Divide then conquer.

To add to RG's list:

When you hear:                                     think about:
squeal                                                    positive feedback
noise                                                      shielding
ticking                                                    layout/shielding/schematic
radio                                                      lack of stoppers/RF suppression
things you can't otherwise explain        fake/blown parts
does nothing (MCU)                               not programmed

[R.G.]

Troubleshooting pedals is just one more area of expertise, no different than automotive repair, radiology, or responding to mystery baskets on the Iron Chef.  A novice may dutifully abide by the protocol laid put here - http://www.diystompboxes.com/wiki/index.php?title=Debugging - and follow it algorithmically, step-by-step, without any sense of priorities.  That's not "wrong", just tedious and frustrating for many.  As R.G.'s post illustrates, over time, and with experience, one comes to know the shortcuts, and the higher probability pairings of observed malfunctions, likely causes, and most diagnostic indicators (i.e., signs to look for).

Good insight, Mark. Brings to mind computer chess playing programs. The best humans beat the best programs as long as the programs were restricted to exhaustive-search and had time limits. When alpha-beta pruning was introduced, it let the computers prune off branches that always got worse, and vastly reduced the search space for solutions. From then on, the computers started beating the best humans.

And correct - the algorithm in "what to do" was intended both to tell absolute beginners what to do, and to force people with small experience out of getting stuck in "losing" branches of the search tree.

... and, well, OK, to tell ME what I needed to know and would wind up asking for in most cases. 

[Mark Hammer]

Chess programs are the prototypic example of how researchers take understanding of expertise, and turn it into software routines/strategies.  Part of that is because some of the earliest, and most thorough, research on expertise was conducted with chess experts and novices.  Chess experts were selected for study not just because we think of serious chess players as "brainy", but because there are formal standards in place for ranking chess players internationally so that researchers could confidently identify a clear group of "experts" who were acknowledged to be ranked higher than other folks with medium or low skill in the area.

And FYI, folks had originally thought that chess experts somehow "looked farther ahead" (in terms of moves) than novices.  Apparently, they don't.  They just know which moves matter more.  Again, using knowledge to carve off a big swathe of what to worry abut and what NOT to be distracted by.

One has to sheepishly admit that "words" only take one so far with debugging.  Some folks here are blessed with the knowledge, the words, AND the equipment.  Some have a meter and nothing else.  And some don't even have a meter.  Using words to trigger what to look for still requires one to have a means of measuring what it is they are looking for.

That doesn't mean that one is necessarily screwed if you don't have a 20ft fully-equipped workbench.  It just means you have to look for other sorts of indicators.

Sometimes the indicators are "naked eye" things.  Sometimes they are change-a-part-and-see-what-happens things.

To add to the list: pops and ticks often connect to sudden charge-up or sudden discharge, producing an audible spike on the power line,

[Paul Marossy]

Good insight, Mark. Brings to mind computer chess playing programs. The best humans beat the best programs as long as the programs were restricted to exhaustive-search and had time limits. When alpha-beta pruning was introduced, it let the computers prune off branches that always got worse, and vastly reduced the search space for solutions. From then on, the computers started beating the best humans.

Man, those computer chess games just slaughter me! (where's a black eye emoticon when you need one?)

[Electron Tornado]

I think that list by R.G. might find its way onto the wall by the workbench.

I'm just a novice, but if I may, I'll add another item to the list:

- Knowing how a system is supposed to correctly operate.

It's kind of the other side of the coin regarding troubleshooting.

[aron]

Loud hum = DC through signal

[oldschoolanalog]

Change a part and see what happens = "Easter Egg Hunt" 

[PRR]

> To add to RG's list:
noise     ----   shielding

I would *BAN* the word "noise".

Well, hand-out a cheat-sheet to *categorize* this "noise".

Hisss
Humm
Buzzz
Radio stations
Squeals
Clicks
Pops
Frying-pan
Popcorn


Fer example, to _me_ the word "noise" implies "hiss". (The other problems have discrete causes and 100% solutions; random noise hiss is at the root of the universe so its properties and optimization fills whole books.)

And generally hiss does not come from bad shielding... that gives hum, buzz, sometimes squeal.



> using knowledge to carve off a big swathe of what to worry about and what NOT to be distracted by.

Yes, I see this in circuit design. Some folks get hung-up on 13-place computation of one bias resistor. I "see" it isn't critical and push-on to a thumb-broad approximate design, see it could be better (whatever that bias resistor was), and quickly move to another hasty plan. Sometimes I see that whole chunks of the problem "don't matter much" and can be dropped-out, or at least left for later. I then have a few-K resistor, a dozens-K resistor, a film cap, a cap that probably has to be electrolytic, medium voltage here, teeny voltage there, a several-mA current consumption. I know it will work, and what ratios matter. Then it's just picking any one part and working out the other values.

[LucifersTrip]

It's great to see additional ideas & techniques that can be incorporated into the debugging thread...

I'd like to add a big one to the list that I don't think has been mentioned

When you hear:            think about:
hum                           power supply
hiss                            power supply
degraded tone             power supply

A bad, unregulated or even a good (in many vintage effects) wallwart is the root of much evil.

...and a more specific one

When you hear:            think about:
hiss                            your leaking ge transistor
noise*                        your leaking ge transistor

*sorry PRR. I don't know how to describe the "shhhhhhhhhhhhhhhhh" and/or intermittent light crackle of a leaky ge

[markeebee]

Ironic that the forum's filter won't let me post the powerful words I use most often when debugging.

Actually, on verified circuits, in my experience at least 99% of the time:

Soldering is fine = soldering is not fine

[Perrow]

Loud square wave sounding signal = you're audio probing the wrong pin of your PT2399

[R.G.]

noise*                        your leaking ge transistor
*sorry PRR. I don't know how to describe the "shhhhhhhhhhhhhhhhh" and/or intermittent light crackle of a leaky ge

You're doing fine. Thermal noise is closest to the sound of escaping steam or compressed air by many descriptions. The technical term for the intermittent crackle is "popcorn noise" or "burst noise". These are usually associated with surface contamination phenomena, and that is often the case in old Ge transistors where modern surface passivation techniques were not available.

[defaced]

> To add to RG's list:
noise     ----   shielding

I would *BAN* the word "noise".

Well, hand-out a cheat-sheet to *categorize* this "noise".

Hisss
Humm
Buzzz
Radio stations
Squeals
Clicks
Pops
Frying-pan
Popcorn

The intent was more to put shielding on the list of possible solutions than to dissect all of the sources of noise than could be lessened by additional shielding.  But to the point, from your list, my "noise" = your "buzz".  That's what I hear when I have a lack of shielding problem. 

[bluebunny]

Nice "prequel", R.G.  Much better than George Lucas could manage...   

[midwayfair]

I'll add this one:

Low output or low gain: You've used a K when you should have used an R.

I see this constantly.

[PRR]

> how to describe the "shhhhhhhhhhhhhhhhh"

Pure simple steady "escaping steam or compressed air" (or FM radio between stations) is "hiss". This is fundamental to the universe.

When it also has burst, popcorn, crackle, that's more complicated.

> my "noise" = your "buzz". 

In english, "noise" covers a LOT of sounds. My neighbor makes a lot of noise. One day it is motor-whirr and rock-on-iron racket. Another day it's banging. And then there's his leaf-blower. I need to know the difference to know to cut the cord off his grinder or cut the handles off his hammers or steal his gasoline.

Likewise, in audio diagnosis, buzz squeal pop etc is a next-step to a fix.

> Soldering is fine = soldering is not fine

I've checked everything!!  == {a week later} Q3 was backward (or) solder-bridge

> Low output or low gain: You've used a K when you should have used an R.

I like to confuse orange and red stripes, and I _know_ that I do. For someone with less experience it is even easier to blind-side stuff like that.

[PRR]

R.G. and Mark-- you may enjoy this parallel excerpt on thought process:

The Internet is surely one of the most innovative and fast-moving areas of technology, and one of the best documented. It has no inventor. Instead it has a massive economy of people who have carefully and progressively solved a long series of immediate problems, documented their answers, and made those available to all. The innovative nature of the Internet comes not from a small, select band of Einsteins. It comes from RFCs anyone can use and improve, made by hundreds and thousands of smart, but not uniquely smart, individuals. It comes from open source software anyone can use and improve. It comes from sharing, scale of community, and the continuous accretion of good solutions and disposal of bad ones.

Here thus is my "Theory of Heuristic Innovation":

1. There is an infinite problem/solution terrain.
2. This terrain changes over time according to external conditions.
3. We can only accurately perceive problems we are close to.
4. We can rank the cost/benefit economics of problems using a market for solutions.
5. There is an optimal solution to any solvable problem.
6. We can approach this optimal solution heuristically, and mechanically.
7. Our intelligence can make this process faster but does not replace it.

It's an approximation. Feel free to send me patches. There are a few takeaways from this:

Individual creativity matters less than process. Smarter people may work faster but they may work in the wrong direction. It's the collective vision of reality that keeps us honest and relevant.

We don't need road-maps if we have a good process. Functionality will emerge and evolve over time as solutions compete for market share.

We don't invent solutions, so much as discover them. All sympathies to the creative soul. It's just an information processing machine that likes to polish its own ego and collect karma.

Intelligence is a social effect, though it feels personal. A person cut-off from others eventually stops thinking.We can neither collect problems nor measure solutions without other people.

The size and diversity of the community is a key factor. Larger, more diverse communities collect more relevant problems, and solve them more accurately, and do this faster, than a small expert group.

It's an after-thought for software, but people is people whatever tools they swing.
http://zguide.wdfiles.com/local--files/main:_start/zguide-php.pdf  page 328