Revision 28 - Version 9.9

A big day today. Revision 28 has a lot of improvements and additions for interpreter capabilities for better performance.

Major addition for the Remove member of a user object (type of Group). Previous versions use Clear to initiate removing function, checking internal if the object has one pointer only point to it.

Now we use the standard method, the original terminate call. Also previous version work only fot pointers to groups, and now works for any group, except the named groups, which by design we didn't want an automatic way to call the remove member (a named group is like a static object, and members of group are actual bound to standard lists for process, so maybe this list erase some before the terminate call to group.

To understand the difficulty, a closed group is an object which hold all members (names and values), and when we use them, all members bound to standard lists from where interpreter use them. So after the use the members get off the lists and placed to closed group. So when the system want to call deconstruction function, and we have a remove member to handle it with our care, we have to open it, link it to standard lists, call the function, and maybe this time another group make a deconstruction too from this deconstruction function. So maybe we have a cascade phenomenon. So here is the difficult point: the standard lists. There are two lists, one for variables/arrays and a second for functions and modules (these are like functions). These are global lists. When a module start to run check the length of those lists, and at the end of execution (at return point) the lists have to delete up to the marked level. This deletion de-reference all internal pointers to objects, one by one from the last to the first in the range of items to delete. So if a Group has a Remove member and an internal flag allow to call it, the list add items, as the Group, from closed status pass to open one. Then the calling to remove function happen so maybe we add more items to list, and at the exit of this functions we have to delete some (but remember we are in the deletion of a module). So now you have the half picture. The list for variables has two arrays, dynamically  added and removed items, always like a LIFO, so the last created variable erased first, and at some points may need to expand it  or reduce it, and that moment we have  a rehashing for the hash table.

I have to say that the solution was easy (which other times I was thought that was impossible or very difficult to address it).  First we have to understand that the arrays are SAFEARRAY type. This type belong to Win32 core. When we use them with variants as items, at the deconstruction process all variants which have pointers to objects to decrement the pointer counter. If any of those counter reach Zero the object enter to deconstruction phase, and the objects which M2000 have also call the terminate part if we have one, and now from that part we call the interpreter. The name SafeArray was for a reason. This array may locked and unlocked. So if we have a locked array we can't reduce it or expand it. If we place an item of array by reference the Visual Basic 6 or any other language which use them, lock the array and unlock it at the return from the call using an internal number to add any number of locks and need the same number for unlocks. To not have locks for references, we have to place pointer to SAFEARRAY not pointer to item. Also we to make the erasing by code, getting a pointer to object, then erase the array item from where we get the pointer, so now we erase the pointer outside from array. Erasing the pointer outside may or may not activate de-construction depends of how many pointers point to same object. So the first problem of locking get a solution. The second problem is for the hash table and the "toplim" value. If we have a cascade phenomenon, each one may expand the two arrays, the item array and the hash table. The solution was to include a static counter in the reducing member in the list object (where the two arrays exist). The reduce member has two parts. the first one has loop from a starting value to an end value, and that end value minus one became a new toplim pointer. For all the of this loop we never read the current toplim, The second part check if the we need to reduce the arrays according to toplim. So before that part we check the counter if it is equal 1 and if it is we proceed to check if we reduce the arrays to half.  So all the job happen in the first part where the objects erased, erasing the pointers (but as we know now, erasing a pointer isn't just writing a null, but a call to a function which erase the pointer by calling first to reduce the reference counter, and then place the null - or nothing). When the reference counter "decide" to terminate the owner (the This object which own the counter).and if we have a remove member in our Group object in the higher level, in M2000, then we start to call one or more functions through the remove member and the toplim start to change, and maybe the arrays start to doubled in size. So lets think now: In the first time toplim was at 10 and we have to place it to 5, so we have to clear from 10 to 6, checking if we have an object and just play with the adding of copy outside the array, and the erase it, to not lock the array. But in 7 item was an object Group which all the flags to call the remove member. We never update topilim until we erase all. so the remove function start from toplim and insert variables as needed. So from 10 maybe became 20, and then at reducing erase from 20 to 11 and set back toplim to 10, but skip the second  part which reduce to half the arrays, when the toplim is lower from the half size of arrays (exactly the first one). So when the call return to first reduce, we have a toplim at 10, and erased down to 11. Also the reducing happen now to later stages because the reduce second part make from current size to half current, and maybe we have one or two times expand. The other reducing may happen later at the checking point in the second part. So now think about the fault, if we let the second reduce in the cascade call (a re-entrance),  if the second decide to reduce, then the first part of the previous call going to check invalid array items (items that we erase at reducing the.array because a safe array when reduce also make the same erasing of pointers somehow and then release the memory).

Check there https://github.com/M2000Interpreter/Version9/blob/master/Group.cls at 356 line (maybe line number changed some days after this post), to see the code from the statement:

 Private Sub Class_Terminate()

In info.gsb there is this program with 4 modules to check the remove member

module test1 {
    Pen 15 {Print "test1: Deconstructor called automatic"}
    class alfa {
        remove {
            Print "remove"
        }
    }
    Dim A(10)<<Pointer(alfa())
    Print "delete 5"
    Dim A(5)
    Print "Now exit"
}
module test2 {
    Pen 15 {Print "test2: insert 8001 variables"}
    For i=1 to 8000
        local i
    next i
    Print "now make a pointer to point a group"
    class alfa {
        remove {
            for i=1 to 10000
            local i
            next
            print "ok"
        }
    }
    m->alfa()
    Print "now module end and deconstructor make 10001 variables"
}
module test3 {
    Pen 15 {Print "test3: A class with a group and one inner group"}
    class alfa {
        remove {
            clear .zeta
            Print "top removed"
        }
        group zeta {
            remove {
                print "inner group zeta removed"
            }
        }
    }
    Print "We make a new stack for values for immediate use"
    Stack New {
        Push alfa()
        Print "now stack erased"
    }
    Print "The inner .zeta removed when we use Clear .zeta inside top deconstructor"
    Print "Now we make a tuple with one item of class alfa()"
    A=(alfa(),)
    Print "end module so tuple erased, and automatic deconstructor executed"
}
module test4 {
    Pen 15 {Print "test4: A class with a group and one pointer to group"}
    class beta {
        remove {
            Pen 11 {print "group class beta removed"}
        }
    }
    class alfa {
        remove {
            clear .zeta
            Print "top removed"
        }
        zeta=Pointer(Beta())
    }
    Print "we make K as named group, not a pointer"
    K=alfa()
    Print "we can execute top group remove function using Clear K"
    Print "but for the example we leave it as is"
    Print "so expected only the pointer to execute the remove function"
}
test1
test2
test3
test4

For replacing tabs to non brake spaces another shitche introduce the NBS

So from console and for current session, we can do this

switches "-NBS" 

and then we can copy code and place it in a word processor or here in blog, with color and without tabs.

Without this switch the M2000 environment  preserve the tabs by using the pre tag. Also that means monospace font for browsers. In M2000 editor we can use any text font with tabs. We can set the width as number of default character width as the font type return, we say that em but not exactly are em (some times we say em is width of M capital, in typography is a height analogous).