Another Scope Mystery

Alright, so this definitely qualifies as an error from bizzarroland.

Today I'm writing a program in Python that involves a function definition that contains the "range" operator - i.e. the ever-popular


for i in range(len(list)):

Well (stupidly, as it turns out), at some other point in the code I named a dictionary "range" as well only to find that this overwrites the reserved keyword!

No, really. The error I was getting was seriously obscure. Basically, what happened was that I had written a bunch of functions to be called in sequence, and then I wanted to encapsulate that call in a single function (that is, call a function that called all those other functions in sequence). Naturally I assumed the output of my cascade would be the same as calling all the functions individually, so the test was a mere formality - one of those "good" programming habits I've developed. So I first called the functions one at a time, saved the end output of that in a variable, and then called my cascaded function on the same input and ...

Traceback (most recent call last):
File "extractor.py", line 86, in module
   print getKeyValues(subjects, relevantfields)
File "extractor.py", line 64, in getKeyValues
   rng = getDataRange(ls, fields)
File "extractor.py", line 49, in getDataRange
  cand = lookup(item[1:], field)
File "extractor.py", line 38, in lookup
  for i in range(len(ls)):
TypeError: 'dict' object is not callable

Of course the first thought is always that you're modifying something, but I wasn't. None of these calls were destructive in any way - just operations on input data. No updates of global variables invovled.

So it took me a really long time to figure out what was going on, but eventually I realized that I had named the dictionary returned at the end of the series of function calls 'range.' It never occurred to me that Python would so easily allow one of its reserved words to be overwritten. And if it allows reserved words as variable names at all, shouldn't it assume, when in conflicts like this, that reserved word is what's wanted and not the variable name? That is, if it runs into a situation where it looks like the user is trying to ... oh, say, call a dictionary as though it were a function ... shouldn't there be buried somewhere in the Python interpreter's error handling something that goes "Hey, wait a minute - this illegal function call/type error that I'm getting hung up on involves the use of a reserved word! Maybe I should try treating that as though it were the reserved word!"

More to the point, if my call to range is inside a function, why would the global dictionary named "range" get in the way at all? Isn't that the point of the "global" keyword, in the end - to allow such access only when we need it? Well, apparently this bumps up against a subtle point in the way Python handles scope. What seems to be going on is that Python's "global" namespace is really a "module" namespace. After the global namespace, it has local (function) namespace, built-in namespace, and nested namespace, and it checks local namespace first. So in cases of name clashes with global variables, local variables are only local at assignment. If there are no assignment statements in your function for a variable name, Python looks in the module ("global") namespace. And I guess some of the keywords that one would expect to be in the "built-in" namespace are actually just function calls from loaded-in modules. So they don't get recognized as keywords, actually. Sure enough, trying to redefine in doesn't work:

File "scopetest.py", line 3
in = 'redefined'
^
SyntaxError: invalid syntax

So "range," being actually a member function of the list type object (???) isn't in the "built-in" namespace, but rather just in the "module" namespace and can therefore be overwritten! Hackers beware!

I really like Python in general - but if I have one complaint about it, this is certainly it: the scope rules are never entirely clear. Of course, really this is my fault for not knowing the language I'm working with as well as I should. And actually, I suppose I should add that Python's behavior is actually completely correct here: languages should let users redefine (certain) keywords if need be. I guess I just didn't expect this to be allowed in a language as general-public directed as Python. Good to know, if surprising! Repeat in unison: Python's Not Java! (And a damn good thing it's not...)