bug#48871: 27.2; Unusably slow in C# mode

bug#48871: 27.2; Unusably slow in C# mode

[jan via Bug reports for GNU Emacs, the Swiss army knife of text editors]

Hi,
I use emacs, I'm not an expert.
C# mode is slow beyond to the point of being completely unusable. This
seems to have started when I upgraded from emacs 26 to emacs 27.2. The
file is ~220K. At the start of the file, typing takes 3 or 4 secs *per
character* to appear (at the end of the file, instantaneous). It's
forcing me to use visual studio to do all simple text editing and I
don't like that.

If I turn off font lock mode it doesn't improve any.
If I switch to fundamental  mode it's back to it's snappy self.
Turning off Syntactic Indentation and Electric Mode didn't help.
I tried removing all comments to see what happened but no luck.

I did find this quite recent issue
<https://github.com/emacs-csharp/csharp-mode/issues/200> but their
solution <https://github.com/emacs-csharp/csharp-mode/issues/200#issuecomment-739926801>
didn't help because I couldn't find that text in the specified lisp
file so couldn't comment it out.

I looked on the emacs bug list but found nothing matching (not sure I
was doing the search right though).

I can provide the c# file but I'd prefer it to not be made public if possible.

Turns out emacs has a profiler, thought I'd try it. From
<https://www.gnu.org/software/emacs/manual
/html_node/elisp/Profiling.html> did profiler-start, typed some chars
(probably 20 to 30 secs locked up before it was finished, immediately
did profiler-stop, report is this (heavier parts expanded)

- command-execute                                                1121  97%
 - call-interactively                                            1121  97%
  - funcall-interactively                                        1114  96%
   - self-insert-command                                          889  77%
    - c-before-change                                             878  76%
     - mapc                                                       878  76%
      - #<compiled 0x2297aaf>                                     878  76%
       - c-before-change-check-unbalanced-strings                 877  76%
          c-pps-to-string-delim                                   865  75%
        - c-syntactic-re-search-forward                            11   0%
           c-beginning-of-macro                                     4   0%
    - c-after-change                                               11   0%
     - mapc                                                        11   0%
      - #<compiled 0x2297b19>                                      11   0%
         c-after-change-mark-abnormal-strings                       4   0%
       - c-restore-<>-properties                                    4   0%
          c-syntactic-re-search-forward                             3   0%
        - c-forward-<>-arglist                                      1   0%
         - c-forward-<>-arglist-recur                               1   0%
            c-forward-sws                                           1   0%
       - c-change-expand-fl-region                                  2   0%
        - c-fl-decl-end                                             2   0%
         - c-literal-start                                          2   0%
          - c-semi-pp-to-literal                                    2   0%
             c-parse-ps-state-below                                 2   0%
         c-parse-quotes-after-change                                1   0%
   - newline                                                      223  19%
    - self-insert-command                                         223  19%
     - electric-indent-post-self-insert-function                  112   9%
      - indent-according-to-mode                                  112   9%
       - c-indent-line                                            112   9%
        - c-shift-line-indentation                                112   9%
         - c-before-change                                        110   9%
          + mapc                                                  109   9%
            c-restore-string-fences                                 1   0%
         - c-after-change                                           2   0%
          - mapc                                                    2   0%
           - #<compiled 0x2297b19>                                  2   0%
              c-after-change-mark-abnormal-strings                  1   0%
            - c-restore-<>-properties                               1   0%
               c-syntactic-re-search-forward                        1   0%
     - c-before-change                                            110   9%
      - mapc                                                      110   9%
       - #<compiled 0x2297aaf>                                    110   9%
        - c-before-change-check-unbalanced-strings                110   9%
           c-pps-to-string-delim                                  109   9%
           c-syntactic-re-search-forward                            1   0%
     - c-after-change                                               1   0%
      - mapc                                                        1   0%
       - #<compiled 0x2297b19>                                      1   0%
        - c-restore-<>-properties                                   1   0%
           c-syntactic-re-search-forward                            1   0%
   - execute-extended-command                                       2   0%
    - sit-for                                                       2   0%
       redisplay                                                    2   0%
  + byte-code                                                       7   0%
- ...                                                              25   2%
   Automatic GC                                                    25   2%
+ redisplay_internal (C function)                                   5   0%

Not sure what to do.

cheers

jan

bug#48871: 27.2; Unusably slow in C# mode

[Eli Zaretskii]

> Date: Sun, 6 Jun 2021 13:32:09 +0100
> From:  jan via "Bug reports for GNU Emacs,
>  the Swiss army knife of text editors" <bug-gnu-emacs@gnu.org>
> 
> C# mode is slow beyond to the point of being completely unusable. This
> seems to have started when I upgraded from emacs 26 to emacs 27.2. The
> file is ~220K. At the start of the file, typing takes 3 or 4 secs *per
> character* to appear (at the end of the file, instantaneous). It's
> forcing me to use visual studio to do all simple text editing and I
> don't like that.
> [...]
> Turns out emacs has a profiler, thought I'd try it.

Good start, thanks.

> Not sure what to do.

Post an example of a file where typing lags by several seconds, and
let's see what people here can tell about that.

But before that, start "emacs -Q", visit the C# file that gave you
such trouble, and try typing there.  If the lag disappears, then look
for some of your customizations that could explain the slow responses.

bug#48871: 27.2; Unusably slow in C# mode

[jan via Bug reports for GNU Emacs, the Swiss army knife of text editors]

Hi Eli,
had already tried the -Q option. Emacs started but when I tried "M-x
csharp-mode" it didn't recognise it. I tried exactly the same on the
normally-started emacs to check I was entering it correctly and that
did understand it.
I guess the -Q effectively disables some modes? I was surprised.

Yep, customisation may well  be the issue here.

I have a file of equivalent size which should demo the issue but can't
zip it as gmail blocks anything with a zip attached, I can post as
attachment directly but it's 220KBytes, you ok with that on your
mailing list?

cheers

jan


On 06/06/2021, Eli Zaretskii <eliz@gnu.org> wrote:
>> Date: Sun, 6 Jun 2021 13:32:09 +0100
>> From:  jan via "Bug reports for GNU Emacs,
>>  the Swiss army knife of text editors" <bug-gnu-emacs@gnu.org>
>>
>> C# mode is slow beyond to the point of being completely unusable. This
>> seems to have started when I upgraded from emacs 26 to emacs 27.2. The
>> file is ~220K. At the start of the file, typing takes 3 or 4 secs *per
>> character* to appear (at the end of the file, instantaneous). It's
>> forcing me to use visual studio to do all simple text editing and I
>> don't like that.
>> [...]
>> Turns out emacs has a profiler, thought I'd try it.
>
> Good start, thanks.
>
>> Not sure what to do.
>
> Post an example of a file where typing lags by several seconds, and
> let's see what people here can tell about that.
>
> But before that, start "emacs -Q", visit the C# file that gave you
> such trouble, and try typing there.  If the lag disappears, then look
> for some of your customizations that could explain the slow responses.
>

bug#48871: 27.2; Unusably slow in C# mode

[Eli Zaretskii]

> From: jan <rtm443x@googlemail.com>
> Date: Sun, 6 Jun 2021 14:34:19 +0100
> Cc: 48871@debbugs.gnu.org
> 
> had already tried the -Q option. Emacs started but when I tried "M-x
> csharp-mode" it didn't recognise it. I tried exactly the same on the
> normally-started emacs to check I was entering it correctly and that
> did understand it.
> I guess the -Q effectively disables some modes? I was surprised.
> 
> Yep, customisation may well  be the issue here.

So let's try to figure out why "emacs -Q" fails to load csharp-mode.
Since there's no such mode bundled with Emacs, I guess you downloaded
it from somewhere?  Then try this:

  emacs -Q
  M-x load-file RET /path/to/csharp-mode.el RE

The last line assumes that the Lisp file which defines the function
csharp-mode is called csharp-mode.el; if not, change the file name to
fit the reality.  Also, "/path/to/" should be replaced with the actual
absolute file name of the file on your system.

Then say what you tried:

   M-x csharp-mode RET

If that still doesn't work, please show the error messages.  Likely
they will identify packages csharp-mode depends on that you also need
to load with "M-x load-file".

> I have a file of equivalent size which should demo the issue but can't
> zip it as gmail blocks anything with a zip attached, I can post as
> attachment directly but it's 220KBytes, you ok with that on your
> mailing list?

Yes, it's okay.  But let's first try and see whether "emacs -Q"
exhibits the same problem, okay?

bug#48871: 27.2; Unusably slow in C# mode

[jan via Bug reports for GNU Emacs, the Swiss army knife of text editors]

[-- Attachment #1: Type: text/plain, Size: 2205 bytes --]

Hi,
I don't recall installing c# but may well have happened.
From package-list-packages:

csharp-mode        20210328.2004 installed             C# mode derived mode

Which does not say built-in so likely I did. Looking in the unzipped
emacs 27.2 , no relevant *sharp* file in it, and did find it in the
elpa directory, so I guess must have.

Started with -Q.  Did the M-x load-file for csharp-mode.el (FYI also
had to do load-file for csharp-compilation.el before that to make it
happy).

Finally got to load the C# file itself, exactly the same. No faster.

Troublesome C# file attached.

cheers

jan

On 06/06/2021, Eli Zaretskii <eliz@gnu.org> wrote:
>> From: jan <rtm443x@googlemail.com>
>> Date: Sun, 6 Jun 2021 14:34:19 +0100
>> Cc: 48871@debbugs.gnu.org
>>
>> had already tried the -Q option. Emacs started but when I tried "M-x
>> csharp-mode" it didn't recognise it. I tried exactly the same on the
>> normally-started emacs to check I was entering it correctly and that
>> did understand it.
>> I guess the -Q effectively disables some modes? I was surprised.
>>
>> Yep, customisation may well  be the issue here.
>
> So let's try to figure out why "emacs -Q" fails to load csharp-mode.
> Since there's no such mode bundled with Emacs, I guess you downloaded
> it from somewhere?  Then try this:
>
>   emacs -Q
>   M-x load-file RET /path/to/csharp-mode.el RE
>
> The last line assumes that the Lisp file which defines the function
> csharp-mode is called csharp-mode.el; if not, change the file name to
> fit the reality.  Also, "/path/to/" should be replaced with the actual
> absolute file name of the file on your system.
>
> Then say what you tried:
>
>    M-x csharp-mode RET
>
> If that still doesn't work, please show the error messages.  Likely
> they will identify packages csharp-mode depends on that you also need
> to load with "M-x load-file".
>
>> I have a file of equivalent size which should demo the issue but can't
>> zip it as gmail blocks anything with a zip attached, I can post as
>> attachment directly but it's 220KBytes, you ok with that on your
>> mailing list?
>
> Yes, it's okay.  But let's first try and see whether "emacs -Q"
> exhibits the same problem, okay?
>

[-- Attachment #2: expression.cs --]
[-- Type: text/plain, Size: 224343 bytes --]

// For reproducing emacs issue of slow response in C# mode.
// Ignore any swearing. 

using Antlr4.Runtime.Tree;  // faq - need this for ParseTreeProperty
using System;
using System.Collections.Generic;
using System.Text;

using static LDB.LDButils;
using static LDB.TypesBroadly;
using static LDB.InstFullyOrPartly;
using static LDB.errorHandling;
using static LDB.SymTab;

using System.Linq;
using System.Diagnostics.CodeAnalysis;  // where from? why? - todo

using static LDB.globals;


// Caused trouble but was generally worth it; caught a few bugs and
// will probably catch a few more. Yep, definitely worth it!
//
//
// Certain methods/properties (e.g. the name property) were declared
// in abstract superclasses but when I added '#nullable enable' they
// started reporting nullability errors. I could either disable those
// with "= null!", totally defeating the point, or deal with them
// properly by pulling those methods/properties down into the concrete
// subclasses, which was more verbose and in some sense less
// encapsulated. In the main, I chose to do that to get static
// guarantees.
#nullable enable


/*

This module is called expressions because that's where I started, and
SQL at heart is an expression language, however there will be
non-expression constructs such as if and loops, and they go here as
well.

*/

namespace LDB {

	public interface IhandlePrefixes<T>
						where T : LDBRoot {

		public bool isPrefixed { get; }

		public bool isExcessPrefixed { get; }

		// The T in addPrefix/rePrefix/withoutPrefix is because we may be 
		// returning eg. a list of items with prefixes 
		// thus modified, not a single PMQIdent

		public T addPrefix(PMQIdent pfx);

		public T rePrefix(PMQIdent newpfx);

//		public T stripPrefix(PMQIdent id);
	}
	// todo - move this





	public abstract class LDBRoot {
		// Root of everything LDB-ish

		public bool Equals([AllowNull] LDBRoot other) {
			throw new LDBShouldNotBeImplementedException(
									"called Equals on LDBRoot");
		}

		public override
			bool Equals([AllowNull] Object other) {
			throw new LDBShouldNotBeImplementedException(
									"called Equals in Object");
		}

		public override int GetHashCode() {
			throw new LDBShouldNotBeImplementedException(
									"called GetHashCode in LDBRoot");
		}

		public static bool operator ==(LDBRoot lhs,
									   LDBRoot rhs) {
			throw new LDBShouldNotBeImplementedException(
									"called == in LDBRoot");
		}

		public static bool operator !=(LDBRoot lhs,
									   LDBRoot rhs) {
			throw new LDBShouldNotBeImplementedException(
									"called != in LDBRoot");
		}


		public virtual string className {
			// todo - this needs checking it works as I want
			// also rename to ldbclassname -- todo
			get => this.GetType().FullName ?? "(unknown)";
		}


		public virtual string classNameQ =>
			"'" + className + "'";


		public virtual string classNamePlus(string plus) =>
			className + "." + plus;

		public override string ToString() {
			// This implicit to-string conversion crap cost me a few hours, so at least
			// do this & blow up at runtime. Would prefer compile time but it seems
			// not possible.  This turns out to have been a good idea,
			// saved me quite a bit more debugging!
			// <https://stackoverflow.com/questions/15345517/is-it-possible-to-disable-implicit-tostring-call>
			// Will try <https://stackoverflow.com/questions/54354342/how-to-get-an-error-shown-when-using-non-strings-as-strings-instead-of-an-autom/54355115#54355115)
			var msg = "LDBRoot.ToString()";
			throw new LDBToStringImplictConversionException(msg);
		}

		public abstract string toDebugStr();
	}



	public abstract class Ephemeral : LDBRoot {
		// for objects that are not part of the AST
	}



	public interface IWithLineNUm {  // todo - use or remove - todo
		//	public 
	}


	public interface IHasTableAlias {
		// Has a table alias AST obj. May be a ...Missing subtype in which
		// it has no alias given by the user, but alias AST is there. More
		// accurately a TableSource alias but ok
		public abstract TableAlias ta { get; set; }
	}


	// Base class for all LDB AST objects
	public abstract class LDBASTobj : LDBRoot {
		// This one is a problem. If I declare the 'parent' property
		// abstract I then have to implement it literally hundreds of
		// times in the derived classes, and I really don't think it
		// would help so just set it to a default which gets overwritten later.
		// 
		// The problem is that those objects were created by the
		// parser, which creates them bottom-up, which means you can't
		// know the parent when they are created. It should be
		// possible to work around this but it does mean reworking the
		// parser fundamentally, and I'm not going to do that
		// assumeing it's even possible.

		// todo - read up on priv/pub/prog accessors I really don't
		// get these at all
		public LDBASTobj parent { get; private set; }
						= noParent;
		// Unique ID per object, useful for distinguishing them
		// without general equality.
		readonly public ulong uid = newUID();

		// public abstract int lineNum { get; set; }

		// Is validated is a cheapish check to ensure everything gets
		// validated
		// can do this as auto prop? todo
		private bool _validated = false;

		protected bool validated {
			get => _validated;
			set {
				// hardMust(!validated, "... already validated... "); - no
				// It's hard to avoid validating a thing twice, so for now allow it
				// Look into it later, really shouldn't happen
				_validated = true;
			}
		}

		public LDBStructsChildren kids { get; private set; } =
					new LDBStructsChildren(noParent); // just easier

		public LDBASTobj() {  //////////////////////////////////////////////////////////////// todo dead?
							  // VStudio thinks this has zero references, which is true explicitly, but it's 
							  // called implicitly on every obj creation - or should do todo- check
			if (recordObjectsCreated) {
				AllCreatedItems.Add(this);
			}
		}


		public virtual void validate() {
			if (reportAlsoIfValidationOK) {
				debugmsg("validating..." + className
					+ " (" + asNameOrIdentContents() + ")");
			}
			mhp_mhk();
			kids.validate();
			// validated = true; -- No, get each subclass to set
			// validated. That forces an explicit call to base() which
			// is more code but ok.
		}


		public bool isValidated { get => validated; }  // todo just => validated;


		public virtual bool isTopLevelItemSeq => false;


		public virtual void mhp_mhk() {  // don't think this need be virtual - todo?
										 // = must have parent, must have kids. The kids may be the
										 // empty set and the parent may be noParent but they must
										 // exist.
										 // Why not just inline this into validate()?
										 // Because a few classes need bespoke validation, but still must check this.
			hardMustParentIsPresent();
			kids.eachKidParentMustBe(this);
		}


		public string asNameOrIdentContents() {
			// for very crude debugging 
			var optName = this is IisNamedItem ni ? ni.name.toRawStr() : "";
			optName = (optName == "" && this is PMQIdent) ? this.toRawStr() : "";

			var optName2 = optName + "  (uid:" + uid.ToString() + ") "
				+ getFirstFewCharsOf(toRawStr());
			return optName2;
		}


		public virtual void getAllUIDs(UIDset uids) {
			uids.Add(this);
			kids.ForEach(namedKid => namedKid.kid.getAllUIDs(uids));
		}


		public bool isStubParent { get => (this is NoParent) || (this is FakeParent); }


		public void validateSpecificItems(params LDBASTobj[] items) {
			Array.ForEach(items, item => item.validate());
		}


		public T? optClimbTo<T>() where T : LDBASTobj {
			var resItem = this;
			while (!(resItem is T)) {
				if (resItem.parent.isStubParent) {
					return null;
				}
				resItem = resItem.parent;
			}

			var res = resItem as T;
			return res;
		}


		public T reqClimbTo<T>() where T : LDBASTobj {
			var loc = className + ".reqClimbTo<T>()";
			var res = optClimbTo<T>();
			hardMust2(!(res is null), 
				// Thanks SO <https://stackoverflow.com/questions/2581642/how-do-i-get-the-type-name-of-a-generic-type-argument>
				// Todo - use this elsewhere, poss ExpectException? todo
				() =>
				loc 
				+ nl + ", climbed to top, didn't find type "
				+ typeof(T).FullName
				+ nl + "Hierarchy is: "
				+ nl + getItemWithParents(this));
			if (res is null) { throw new NNQC(); }
			return res;
		}


		public void hardMustParentIsPresent() {

			hardMust2(!(parent is null), 
				() =>
				 "NULL parent for item of type\n"
				 + $"{GetType()}\n.");

			hardMust2(!(this.parent is NoParent),
				() =>
				"Parent is NoParent on object of type\n"
				+ $"'{this.className}'");
		}


		public void hardMustParentIsAbsent() {
			// For checking cloning - parent should be NoParent immediately 
			// after, before it's set by parent, and never null
			hardMust(!(parent is null),
				 "NULL parent for item of type\n"
				 + $"{GetType()}\n.");
			hardMust((this.parent is NoParent),
				"Parent should be NoParent on object of type\n"
				+ $"'{this.className}'");
		}


		public bool hasParent<P>() {
			hardMust(!(this is NoParent), "Item is NoParent");
			return this.parent is P;
		}


		public bool hasAncestor<A>() {
			hardMust(!this.isStubParent, "Item is NoParent");

			// how to do with a switch expr?
			if (this.parent.isStubParent /* is NoParent || this.parent is FakeParent*/ ) { // todo do as x.hasFudgedParent() - todo
				return false;
			} else if (this.parent is A) {
				return true;
			} else {
				return this.parent.hasAncestor<A>();
			}
		}

		public bool notHasAncestor<A>() =>
			!hasAncestor<A>();


		public void setMyParentTo(LDBASTobj prnt) {
			// An object may only set its parent if its current parent is
			// the dummy 'noParent', which almost every object's parent is
			// set to on creation (see LDBASTobj parent property def).

			// The test for null is peculiar given this. Because
			// noParent is created statically, setting the parent of
			// noParent is difficult, perhaps impossible, to do statically
			// and reliably.Despite the declaration '#nullable enable', it
			// gets created with a null parrent (quite reasonably), hence
			// this check.The actual setting of the parent has to be done
			// at runtime i.e.after the static creation of 'noParent'.

			hardMust2((this.parent is NoParent) || (this.parent is null),
				() =>
				"tried to re-set parent on object of type: "
				+ nl + this.classNameQ
				+ nl + "Original parent type is: "
				+ (this?.parent?.classNameQ == null ? "null" : this.parent.classNameQ)
				+ nl + "and new parent type is: "
				+ prnt.classNameQ
				+ nl + "Item having parent set on it is: "
				// because object may not be fully constructed yet, check howver TODO this is a major hack and must be fixed sometime.
//				+ nl + (this is null ? "(null)" : this.toRawStr()));
				+ nl + this.toRawStr() );

			parent = prnt;
		}


		public void forceNewParentForRootObject() {
			// Every object must have some parent, including the
			// root object of the parse, which is LDBitems
			// (a sequence of LDB items), so this forces one.
			// It is only for that purpose!
			// 
			// LDBitems obj exists for sub objs such as the statements
			// in a while loop, in a proc, in an if... etc., for
			// which their parent is the while/proc/if etc.
			hardMustParentIsAbsent();
			this.parent = fakeParent;
		}


		//public void resetMyParentTo(LDBASTobj prnt) {  ////////////////////////////////// destroy!!!!!! todo
		//	parent = prnt;
		//}

		//////////////////////// xxxxxxxxxxxxx delete - todo
		//private void setParentOnKid(LDBASTobj kid) => // todo needed?
		//											  // check parente doesn't exist - todo
		//	kid.parent = this;

		public virtual void setParentOnKidsAndMakeKidsDict(  // todo - and add to kids dict
															 // this mixes up setting parent and making child dict. Not good. - todo
						params (string kidName, LDBASTobj kid)[] kidsIn) {
			var loc = classNamePlus("setParentOnKidsAndMakeKidsDict");

			var kidsToAdd = new LDBStructsChildren(this, kidsIn);
			hardMust2(kids.isEmpty,
				() =>
				$"setting kids on parent '{className}'" 
				+ "when kids is not empty, "
				+ $"kids is ({kids.Count} kids):"
				+ nl + kids.toDebugStr()
				+ nl + $"Trying to add\n{kidsToAdd.toDebugStr()}\n"
				+ $"in {className}");
			// if there is an LDBStructsChildren and another child,
			// this will blow up - will need to do kids.Add or
			// something - in fact, kids needs to be created in one
			// go, which is what happens.
			kids = kidsToAdd;    ///////////////////////////////////////////////////////// why this? todo
			Array.ForEach(kidsIn, xkid => hardMust(!(xkid.kid is null), loc, "null kid"));
			Array.ForEach(kidsIn, xkid => xkid.kid.setMyParentTo(this));
		}


		public virtual void setParentOnKidsAndAddToKidsDict(
					params (string kidName, LDBASTobj kid)[] kidsIn) {
			var kidsToAdd_justForErrReporting = new LDBStructsChildren(this, kidsIn);
			hardMust2(!kids.isEmpty, // may disable this later 
				() =>
				"adding to kids when kids is empty, "
				 + $"kids is\n{kids.toDebugStr()}\n"
				 + $"Trying to add\n{kidsToAdd_justForErrReporting.toDebugStr()}\n"
				 + $"in {className}");
			Array.ForEach(kidsIn, xkid => {
//				setParentOnKid(xkid.kid);
				xkid.kid.setMyParentTo(this);
				kids.Add(xkid);
			});
		}


		public virtual IdentsUsed getAllIDs() {
		/* This is overridden only in TemplatedAgg, and that's only to 
			deliberately fail. There's something wrong there, todo - 
			un-virtualise this and remove the one override. - todo
		*/
			var iu = new IdentsUsed();
			this._getAllIDs(iu);
			return iu;
		}

		public virtual void _getAllIDs(IdentsUsed iu) {
		// Gets all the identifiers explicitly used (read or written)  ----- todo update this
		// anywhere in the AST.  Does not get idents which are
		// declarations ie. parameters, or var/const declarations
		// unless these are combined with an assignment.
		//
		// Implied idents such as those from tbl.* are implicit and ignored.
		//
		// Root caller supplies new, empty iu
		// need usage here - todo docs//
			kids._getAllIDs(iu);
		}

		public abstract bool canGetAllIdentsUsed {
		// 	This seems to be true on most classes so I could set true 
		// as a default here and override it where it's not wanted, saving 
		// a chunk of code, but I'd rather make it explicit to force me to consider each case. A bit more C&P for less chance of bugs.
			get;
		}


		// Produces a string that should be identical to (excepting
		// whitespaces) the input ie. emit LDB code not valid SQL
		//
		// ToString was causing too many weird problems (due to my
		// inexperience with c#) so just used toRawStr()
		public abstract string toRawStr();

		public override string toDebugStr() => toRawStr();

	} // end LDBASTobj



	public interface IhandlePrefixes<T>
						where T : LDBRoot {

		public bool isPrefixed { get; }

		public bool isExcessPrefixed { get; }

		// The T in addPrefix/rePrefix/withoutPrefix is because we may be 
		// returning eg. a list of items with prefixes 
		// thus modified, not a single PMQIdent

		public T addPrefix(PMQIdent pfx);

		public T rePrefix(PMQIdent newpfx);

//		public T stripPrefix(PMQIdent id);
	}
	// todo - move this





	public abstract class LDBRoot {
		// Root of everything LDB-ish

		public bool Equals([AllowNull] LDBRoot other) {
			throw new LDBShouldNotBeImplementedException(
									"called Equals on LDBRoot");
		}

		public override
			bool Equals([AllowNull] Object other) {
			throw new LDBShouldNotBeImplementedException(
									"called Equals in Object");
		}

		public override int GetHashCode() {
			throw new LDBShouldNotBeImplementedException(
									"called GetHashCode in LDBRoot");
		}

		public static bool operator ==(LDBRoot lhs,
									   LDBRoot rhs) {
			throw new LDBShouldNotBeImplementedException(
									"called == in LDBRoot");
		}

		public static bool operator !=(LDBRoot lhs,
									   LDBRoot rhs) {
			throw new LDBShouldNotBeImplementedException(
									"called != in LDBRoot");
		}


		public virtual string className {
			// todo - this needs checking it works as I want
			// also rename to ldbclassname -- todo
			get => this.GetType().FullName ?? "(unknown)";
		}


		public virtual string classNameQ =>
			"'" + className + "'";


		public virtual string classNamePlus(string plus) =>
			className + "." + plus;

		public override string ToString() {
			// This implicit to-string conversion crap cost me a few hours, so at least
			// do this & blow up at runtime. Would prefer compile time but it seems
			// not possible.  This turns out to have been a good idea,
			// saved me quite a bit more debugging!
			// <https://stackoverflow.com/questions/15345517/is-it-possible-to-disable-implicit-tostring-call>
			// Will try <https://stackoverflow.com/questions/54354342/how-to-get-an-error-shown-when-using-non-strings-as-strings-instead-of-an-autom/54355115#54355115)
			var msg = "LDBRoot.ToString()";
			throw new LDBToStringImplictConversionException(msg);
		}

		public abstract string toDebugStr();
	}



	public abstract class Ephemeral : LDBRoot {
		// for objects that are not part of the AST
	}



	public interface IWithLineNUm {  // todo - use or remove - todo
		//	public 
	}


	public interface IHasTableAlias {
		// Has a table alias AST obj. May be a ...Missing subtype in which
		// it has no alias given by the user, but alias AST is there. More
		// accurately a TableSource alias but ok
		public abstract TableAlias ta { get; set; }
	}


	// Base class for all LDB AST objects
	public abstract class LDBASTobj : LDBRoot {
		// This one is a problem. If I declare the 'parent' property
		// abstract I then have to implement it literally hundreds of
		// times in the derived classes, and I really don't think it
		// would help so just set it to a default which gets overwritten later.
		// 
		// The problem is that those objects were created by the
		// parser, which creates them bottom-up, which means you can't
		// know the parent when they are created. It should be
		// possible to work around this but it does mean reworking the
		// parser fundamentally, and I'm not going to do that
		// assumeing it's even possible.

		// todo - read up on priv/pub/prog accessors I really don't
		// get these at all
		public LDBASTobj parent { get; private set; }
						= noParent;
		// Unique ID per object, useful for distinguishing them
		// without general equality.
		readonly public ulong uid = newUID();

		// public abstract int lineNum { get; set; }

		// Is validated is a cheapish check to ensure everything gets
		// validated
		// can do this as auto prop? todo
		private bool _validated = false;

		protected bool validated {
			get => _validated;
			set {
				// hardMust(!validated, "... already validated... "); - no
				// It's hard to avoid validating a thing twice, so for now allow it
				// Look into it later, really shouldn't happen
				_validated = true;
			}
		}

		public LDBStructsChildren kids { get; private set; } =
					new LDBStructsChildren(noParent); // just easier

		public LDBASTobj() {  //////////////////////////////////////////////////////////////// todo dead?
							  // VStudio thinks this has zero references, which is true explicitly, but it's 
							  // called implicitly on every obj creation - or should do todo- check
			if (recordObjectsCreated) {
				AllCreatedItems.Add(this);
			}
		}


		public virtual void validate() {
			if (reportAlsoIfValidationOK) {
				debugmsg("validating..." + className
					+ " (" + asNameOrIdentContents() + ")");
			}
			mhp_mhk();
			kids.validate();
			// validated = true; -- No, get each subclass to set
			// validated. That forces an explicit call to base() which
			// is more code but ok.
		}


		public bool isValidated { get => validated; }  // todo just => validated;


		public virtual bool isTopLevelItemSeq => false;


		public virtual void mhp_mhk() {  // don't think this need be virtual - todo?
										 // = must have parent, must have kids. The kids may be the
										 // empty set and the parent may be noParent but they must
										 // exist.
										 // Why not just inline this into validate()?
										 // Because a few classes need bespoke validation, but still must check this.
			hardMustParentIsPresent();
			kids.eachKidParentMustBe(this);
		}


		public string asNameOrIdentContents() {
			// for very crude debugging 
			var optName = this is IisNamedItem ni ? ni.name.toRawStr() : "";
			optName = (optName == "" && this is PMQIdent) ? this.toRawStr() : "";

			var optName2 = optName + "  (uid:" + uid.ToString() + ") "
				+ getFirstFewCharsOf(toRawStr());
			return optName2;
		}


		public virtual void getAllUIDs(UIDset uids) {
			uids.Add(this);
			kids.ForEach(namedKid => namedKid.kid.getAllUIDs(uids));
		}


		public bool isStubParent { get => (this is NoParent) || (this is FakeParent); }


		public void validateSpecificItems(params LDBASTobj[] items) {
			Array.ForEach(items, item => item.validate());
		}


		public T? optClimbTo<T>() where T : LDBASTobj {
			var resItem = this;
			while (!(resItem is T)) {
				if (resItem.parent.isStubParent) {
					return null;
				}
				resItem = resItem.parent;
			}

			var res = resItem as T;
			return res;
		}


		public T reqClimbTo<T>() where T : LDBASTobj {
			var loc = className + ".reqClimbTo<T>()";
			var res = optClimbTo<T>();
			hardMust2(!(res is null), 
				// Thanks SO <https://stackoverflow.com/questions/2581642/how-do-i-get-the-type-name-of-a-generic-type-argument>
				// Todo - use this elsewhere, poss ExpectException? todo
				() =>
				loc 
				+ nl + ", climbed to top, didn't find type "
				+ typeof(T).FullName
				+ nl + "Hierarchy is: "
				+ nl + getItemWithParents(this));
			if (res is null) { throw new NNQC(); }
			return res;
		}


		public void hardMustParentIsPresent() {

			hardMust2(!(parent is null), 
				() =>
				 "NULL parent for item of type\n"
				 + $"{GetType()}\n.");

			hardMust2(!(this.parent is NoParent),
				() =>
				"Parent is NoParent on object of type\n"
				+ $"'{this.className}'");
		}


		public void hardMustParentIsAbsent() {
			// For checking cloning - parent should be NoParent immediately 
			// after, before it's set by parent, and never null
			hardMust(!(parent is null),
				 "NULL parent for item of type\n"
				 + $"{GetType()}\n.");
			hardMust((this.parent is NoParent),
				"Parent should be NoParent on object of type\n"
				+ $"'{this.className}'");
		}


		public bool hasParent<P>() {
			hardMust(!(this is NoParent), "Item is NoParent");
			return this.parent is P;
		}


		public bool hasAncestor<A>() {
			hardMust(!this.isStubParent, "Item is NoParent");

			// how to do with a switch expr?
			if (this.parent.isStubParent /* is NoParent || this.parent is FakeParent*/ ) { // todo do as x.hasFudgedParent() - todo
				return false;
			} else if (this.parent is A) {
				return true;
			} else {
				return this.parent.hasAncestor<A>();
			}
		}

		public bool notHasAncestor<A>() =>
			!hasAncestor<A>();


		public void setMyParentTo(LDBASTobj prnt) {
			// An object may only set its parent if its current parent is
			// the dummy 'noParent', which almost every object's parent is
			// set to on creation (see LDBASTobj parent property def).

			// The test for null is peculiar given this. Because
			// noParent is created statically, setting the parent of
			// noParent is difficult, perhaps impossible, to do statically
			// and reliably.Despite the declaration '#nullable enable', it
			// gets created with a null parrent (quite reasonably), hence
			// this check.The actual setting of the parent has to be done
			// at runtime i.e.after the static creation of 'noParent'.

			hardMust2((this.parent is NoParent) || (this.parent is null),
				() =>
				"tried to re-set parent on object of type: "
				+ nl + this.classNameQ
				+ nl + "Original parent type is: "
				+ (this?.parent?.classNameQ == null ? "null" : this.parent.classNameQ)
				+ nl + "and new parent type is: "
				+ prnt.classNameQ
				+ nl + "Item having parent set on it is: "
				// because object may not be fully constructed yet, check howver TODO this is a major hack and must be fixed sometime.
//				+ nl + (this is null ? "(null)" : this.toRawStr()));
				+ nl + this.toRawStr() );

			parent = prnt;
		}


		public void forceNewParentForRootObject() {
			// Every object must have some parent, including the
			// root object of the parse, which is LDBitems
			// (a sequence of LDB items), so this forces one.
			// It is only for that purpose!
			// 
			// LDBitems obj exists for sub objs such as the statements
			// in a while loop, in a proc, in an if... etc., for
			// which their parent is the while/proc/if etc.
			hardMustParentIsAbsent();
			this.parent = fakeParent;
		}


		//public void resetMyParentTo(LDBASTobj prnt) {  ////////////////////////////////// destroy!!!!!! todo
		//	parent = prnt;
		//}

		//////////////////////// xxxxxxxxxxxxx delete - todo
		//private void setParentOnKid(LDBASTobj kid) => // todo needed?
		//											  // check parente doesn't exist - todo
		//	kid.parent = this;

		public virtual void setParentOnKidsAndMakeKidsDict(  // todo - and add to kids dict
															 // this mixes up setting parent and making child dict. Not good. - todo
						params (string kidName, LDBASTobj kid)[] kidsIn) {
			var loc = classNamePlus("setParentOnKidsAndMakeKidsDict");

			var kidsToAdd = new LDBStructsChildren(this, kidsIn);
			hardMust2(kids.isEmpty,
				() =>
				$"setting kids on parent '{className}'" 
				+ "when kids is not empty, "
				+ $"kids is ({kids.Count} kids):"
				+ nl + kids.toDebugStr()
				+ nl + $"Trying to add\n{kidsToAdd.toDebugStr()}\n"
				+ $"in {className}");
			// if there is an LDBStructsChildren and another child,
			// this will blow up - will need to do kids.Add or
			// something - in fact, kids needs to be created in one
			// go, which is what happens.
			kids = kidsToAdd;    ///////////////////////////////////////////////////////// why this? todo
			Array.ForEach(kidsIn, xkid => hardMust(!(xkid.kid is null), loc, "null kid"));
			Array.ForEach(kidsIn, xkid => xkid.kid.setMyParentTo(this));
		}


		public virtual void setParentOnKidsAndAddToKidsDict(
					params (string kidName, LDBASTobj kid)[] kidsIn) {
			var kidsToAdd_justForErrReporting = new LDBStructsChildren(this, kidsIn);
			hardMust2(!kids.isEmpty, // may disable this later 
				() =>
				"adding to kids when kids is empty, "
				 + $"kids is\n{kids.toDebugStr()}\n"
				 + $"Trying to add\n{kidsToAdd_justForErrReporting.toDebugStr()}\n"
				 + $"in {className}");
			Array.ForEach(kidsIn, xkid => {
//				setParentOnKid(xkid.kid);
				xkid.kid.setMyParentTo(this);
				kids.Add(xkid);
			});
		}


		public virtual IdentsUsed getAllIDs() {
		/* This is overridden only in TemplatedAgg, and that's only to 
			deliberately fail. There's something wrong there, todo - 
			un-virtualise this and remove the one override. - todo
		*/
			var iu = new IdentsUsed();
			this._getAllIDs(iu);
			return iu;
		}

		public virtual void _getAllIDs(IdentsUsed iu) {
		// Gets all the identifiers explicitly used (read or written)  ----- todo update this
		// anywhere in the AST.  Does not get idents which are
		// declarations ie. parameters, or var/const declarations
		// unless these are combined with an assignment.
		//
		// Implied idents such as those from tbl.* are implicit and ignored.
		//
		// Root caller supplies new, empty iu
		// need usage here - todo docs//
			kids._getAllIDs(iu);
		}

		public abstract bool canGetAllIdentsUsed {
		// 	This seems to be true on most classes so I could set true 
		// as a default here and override it where it's not wanted, saving 
		// a chunk of code, but I'd rather make it explicit to force me to consider each case. A bit more C&P for less chance of bugs.
			get;
		}


		// Produces a string that should be identical to (excepting
		// whitespaces) the input ie. emit LDB code not valid SQL
		//
		// ToString was causing too many weird problems (due to my
		// inexperience with c#) so just used toRawStr()
		public abstract string toRawStr();

		public override string toDebugStr() => toRawStr();

	} // end LDBASTobj



	public interface IhandlePrefixes<T>
						where T : LDBRoot {

		public bool isPrefixed { get; }

		public bool isExcessPrefixed { get; }

		// The T in addPrefix/rePrefix/withoutPrefix is because we may be 
		// returning eg. a list of items with prefixes 
		// thus modified, not a single PMQIdent

		public T addPrefix(PMQIdent pfx);

		public T rePrefix(PMQIdent newpfx);

//		public T stripPrefix(PMQIdent id);
	}
	// todo - move this





	public abstract class LDBRoot {
		// Root of everything LDB-ish

		public bool Equals([AllowNull] LDBRoot other) {
			throw new LDBShouldNotBeImplementedException(
									"called Equals on LDBRoot");
		}

		public override
			bool Equals([AllowNull] Object other) {
			throw new LDBShouldNotBeImplementedException(
									"called Equals in Object");
		}

		public override int GetHashCode() {
			throw new LDBShouldNotBeImplementedException(
									"called GetHashCode in LDBRoot");
		}

		public static bool operator ==(LDBRoot lhs,
									   LDBRoot rhs) {
			throw new LDBShouldNotBeImplementedException(
									"called == in LDBRoot");
		}

		public static bool operator !=(LDBRoot lhs,
									   LDBRoot rhs) {
			throw new LDBShouldNotBeImplementedException(
									"called != in LDBRoot");
		}


		public virtual string className {
			// todo - this needs checking it works as I want
			// also rename to ldbclassname -- todo
			get => this.GetType().FullName ?? "(unknown)";
		}


		public virtual string classNameQ =>
			"'" + className + "'";


		public virtual string classNamePlus(string plus) =>
			className + "." + plus;

		public override string ToString() {
			// This implicit to-string conversion crap cost me a few hours, so at least
			// do this & blow up at runtime. Would prefer compile time but it seems
			// not possible.  This turns out to have been a good idea,
			// saved me quite a bit more debugging!
			// <https://stackoverflow.com/questions/15345517/is-it-possible-to-disable-implicit-tostring-call>
			// Will try <https://stackoverflow.com/questions/54354342/how-to-get-an-error-shown-when-using-non-strings-as-strings-instead-of-an-autom/54355115#54355115)
			var msg = "LDBRoot.ToString()";
			throw new LDBToStringImplictConversionException(msg);
		}

		public abstract string toDebugStr();
	}



	public abstract class Ephemeral : LDBRoot {
		// for objects that are not part of the AST
	}



	public interface IWithLineNUm {  // todo - use or remove - todo
		//	public 
	}


	public interface IHasTableAlias {
		// Has a table alias AST obj. May be a ...Missing subtype in which
		// it has no alias given by the user, but alias AST is there. More
		// accurately a TableSource alias but ok
		public abstract TableAlias ta { get; set; }
	}


	// Base class for all LDB AST objects
	public abstract class LDBASTobj : LDBRoot {
		// This one is a problem. If I declare the 'parent' property
		// abstract I then have to implement it literally hundreds of
		// times in the derived classes, and I really don't think it
		// would help so just set it to a default which gets overwritten later.
		// 
		// The problem is that those objects were created by the
		// parser, which creates them bottom-up, which means you can't
		// know the parent when they are created. It should be
		// possible to work around this but it does mean reworking the
		// parser fundamentally, and I'm not going to do that
		// assumeing it's even possible.

		// todo - read up on priv/pub/prog accessors I really don't
		// get these at all
		public LDBASTobj parent { get; private set; }
						= noParent;
		// Unique ID per object, useful for distinguishing them
		// without general equality.
		readonly public ulong uid = newUID();

		// public abstract int lineNum { get; set; }

		// Is validated is a cheapish check to ensure everything gets
		// validated
		// can do this as auto prop? todo
		private bool _validated = false;

		protected bool validated {
			get => _validated;
			set {
				// hardMust(!validated, "... already validated... "); - no
				// It's hard to avoid validating a thing twice, so for now allow it
				// Look into it later, really shouldn't happen
				_validated = true;
			}
		}

		public LDBStructsChildren kids { get; private set; } =
					new LDBStructsChildren(noParent); // just easier

		public LDBASTobj() {  //////////////////////////////////////////////////////////////// todo dead?
							  // VStudio thinks this has zero references, which is true explicitly, but it's 
							  // called implicitly on every obj creation - or should do todo- check
			if (recordObjectsCreated) {
				AllCreatedItems.Add(this);
			}
		}


		public virtual void validate() {
			if (reportAlsoIfValidationOK) {
				debugmsg("validating..." + className
					+ " (" + asNameOrIdentContents() + ")");
			}
			mhp_mhk();
			kids.validate();
			// validated = true; -- No, get each subclass to set
			// validated. That forces an explicit call to base() which
			// is more code but ok.
		}


		public bool isValidated { get => validated; }  // todo just => validated;


		public virtual bool isTopLevelItemSeq => false;


		public virtual void mhp_mhk() {  // don't think this need be virtual - todo?
										 // = must have parent, must have kids. The kids may be the
										 // empty set and the parent may be noParent but they must
										 // exist.
										 // Why not just inline this into validate()?
										 // Because a few classes need bespoke validation, but still must check this.
			hardMustParentIsPresent();
			kids.eachKidParentMustBe(this);
		}


		public string asNameOrIdentContents() {
			// for very crude debugging 
			var optName = this is IisNamedItem ni ? ni.name.toRawStr() : "";
			optName = (optName == "" && this is PMQIdent) ? this.toRawStr() : "";

			var optName2 = optName + "  (uid:" + uid.ToString() + ") "
				+ getFirstFewCharsOf(toRawStr());
			return optName2;
		}


		public virtual void getAllUIDs(UIDset uids) {
			uids.Add(this);
			kids.ForEach(namedKid => namedKid.kid.getAllUIDs(uids));
		}


		public bool isStubParent { get => (this is NoParent) || (this is FakeParent); }


		public void validateSpecificItems(params LDBASTobj[] items) {
			Array.ForEach(items, item => item.validate());
		}


		public T? optClimbTo<T>() where T : LDBASTobj {
			var resItem = this;
			while (!(resItem is T)) {
				if (resItem.parent.isStubParent) {
					return null;
				}
				resItem = resItem.parent;
			}

			var res = resItem as T;
			return res;
		}


		public T reqClimbTo<T>() where T : LDBASTobj {
			var loc = className + ".reqClimbTo<T>()";
			var res = optClimbTo<T>();
			hardMust2(!(res is null), 
				// Thanks SO <https://stackoverflow.com/questions/2581642/how-do-i-get-the-type-name-of-a-generic-type-argument>
				// Todo - use this elsewhere, poss ExpectException? todo
				() =>
				loc 
				+ nl + ", climbed to top, didn't find type "
				+ typeof(T).FullName
				+ nl + "Hierarchy is: "
				+ nl + getItemWithParents(this));
			if (res is null) { throw new NNQC(); }
			return res;
		}


		public void hardMustParentIsPresent() {

			hardMust2(!(parent is null), 
				() =>
				 "NULL parent for item of type\n"
				 + $"{GetType()}\n.");

			hardMust2(!(this.parent is NoParent),
				() =>
				"Parent is NoParent on object of type\n"
				+ $"'{this.className}'");
		}


		public void hardMustParentIsAbsent() {
			// For checking cloning - parent should be NoParent immediately 
			// after, before it's set by parent, and never null
			hardMust(!(parent is null),
				 "NULL parent for item of type\n"
				 + $"{GetType()}\n.");
			hardMust((this.parent is NoParent),
				"Parent should be NoParent on object of type\n"
				+ $"'{this.className}'");
		}


		public bool hasParent<P>() {
			hardMust(!(this is NoParent), "Item is NoParent");
			return this.parent is P;
		}


		public bool hasAncestor<A>() {
			hardMust(!this.isStubParent, "Item is NoParent");

			// how to do with a switch expr?
			if (this.parent.isStubParent /* is NoParent || this.parent is FakeParent*/ ) { // todo do as x.hasFudgedParent() - todo
				return false;
			} else if (this.parent is A) {
				return true;
			} else {
				return this.parent.hasAncestor<A>();
			}
		}

		public bool notHasAncestor<A>() =>
			!hasAncestor<A>();


		public void setMyParentTo(LDBASTobj prnt) {
			// An object may only set its parent if its current parent is
			// the dummy 'noParent', which almost every object's parent is
			// set to on creation (see LDBASTobj parent property def).

			// The test for null is peculiar given this. Because
			// noParent is created statically, setting the parent of
			// noParent is difficult, perhaps impossible, to do statically
			// and reliably.Despite the declaration '#nullable enable', it
			// gets created with a null parrent (quite reasonably), hence
			// this check.The actual setting of the parent has to be done
			// at runtime i.e.after the static creation of 'noParent'.

			hardMust2((this.parent is NoParent) || (this.parent is null),
				() =>
				"tried to re-set parent on object of type: "
				+ nl + this.classNameQ
				+ nl + "Original parent type is: "
				+ (this?.parent?.classNameQ == null ? "null" : this.parent.classNameQ)
				+ nl + "and new parent type is: "
				+ prnt.classNameQ
				+ nl + "Item having parent set on it is: "
				// because object may not be fully constructed yet, check howver TODO this is a major hack and must be fixed sometime.
//				+ nl + (this is null ? "(null)" : this.toRawStr()));
				+ nl + this.toRawStr() );

			parent = prnt;
		}


		public void forceNewParentForRootObject() {
			// Every object must have some parent, including the
			// root object of the parse, which is LDBitems
			// (a sequence of LDB items), so this forces one.
			// It is only for that purpose!
			// 
			// LDBitems obj exists for sub objs such as the statements
			// in a while loop, in a proc, in an if... etc., for
			// which their parent is the while/proc/if etc.
			hardMustParentIsAbsent();
			this.parent = fakeParent;
		}


		//public void resetMyParentTo(LDBASTobj prnt) {  ////////////////////////////////// destroy!!!!!! todo
		//	parent = prnt;
		//}

		//////////////////////// xxxxxxxxxxxxx delete - todo
		//private void setParentOnKid(LDBASTobj kid) => // todo needed?
		//											  // check parente doesn't exist - todo
		//	kid.parent = this;

		public virtual void setParentOnKidsAndMakeKidsDict(  // todo - and add to kids dict
															 // this mixes up setting parent and making child dict. Not good. - todo
						params (string kidName, LDBASTobj kid)[] kidsIn) {
			var loc = classNamePlus("setParentOnKidsAndMakeKidsDict");

			var kidsToAdd = new LDBStructsChildren(this, kidsIn);
			hardMust2(kids.isEmpty,
				() =>
				$"setting kids on parent '{className}'" 
				+ "when kids is not empty, "
				+ $"kids is ({kids.Count} kids):"
				+ nl + kids.toDebugStr()
				+ nl + $"Trying to add\n{kidsToAdd.toDebugStr()}\n"
				+ $"in {className}");
			// if there is an LDBStructsChildren and another child,
			// this will blow up - will need to do kids.Add or
			// something - in fact, kids needs to be created in one
			// go, which is what happens.
			kids = kidsToAdd;    ///////////////////////////////////////////////////////// why this? todo
			Array.ForEach(kidsIn, xkid => hardMust(!(xkid.kid is null), loc, "null kid"));
			Array.ForEach(kidsIn, xkid => xkid.kid.setMyParentTo(this));
		}


		public virtual void setParentOnKidsAndAddToKidsDict(
					params (string kidName, LDBASTobj kid)[] kidsIn) {
			var kidsToAdd_justForErrReporting = new LDBStructsChildren(this, kidsIn);
			hardMust2(!kids.isEmpty, // may disable this later 
				() =>
				"adding to kids when kids is empty, "
				 + $"kids is\n{kids.toDebugStr()}\n"
				 + $"Trying to add\n{kidsToAdd_justForErrReporting.toDebugStr()}\n"
				 + $"in {className}");
			Array.ForEach(kidsIn, xkid => {
//				setParentOnKid(xkid.kid);
				xkid.kid.setMyParentTo(this);
				kids.Add(xkid);
			});
		}


		public virtual IdentsUsed getAllIDs() {
		/* This is overridden only in TemplatedAgg, and that's only to 
			deliberately fail. There's something wrong there, todo - 
			un-virtualise this and remove the one override. - todo
		*/
			var iu = new IdentsUsed();
			this._getAllIDs(iu);
			return iu;
		}

		public virtual void _getAllIDs(IdentsUsed iu) {
		// Gets all the identifiers explicitly used (read or written)  ----- todo update this
		// anywhere in the AST.  Does not get idents which are
		// declarations ie. parameters, or var/const declarations
		// unless these are combined with an assignment.
		//
		// Implied idents such as those from tbl.* are implicit and ignored.
		//
		// Root caller supplies new, empty iu
		// need usage here - todo docs//
			kids._getAllIDs(iu);
		}

		public abstract bool canGetAllIdentsUsed {
		// 	This seems to be true on most classes so I could set true 
		// as a default here and override it where it's not wanted, saving 
		// a chunk of code, but I'd rather make it explicit to force me to consider each case. A bit more C&P for less chance of bugs.
			get;
		}


		// Produces a string that should be identical to (excepting
		// whitespaces) the input ie. emit LDB code not valid SQL
		//
		// ToString was causing too many weird problems (due to my
		// inexperience with c#) so just used toRawStr()
		public abstract string toRawStr();

		public override string toDebugStr() => toRawStr();

	} // end LDBASTobj



	public interface IhandlePrefixes<T>
						where T : LDBRoot {

		public bool isPrefixed { get; }

		public bool isExcessPrefixed { get; }

		// The T in addPrefix/rePrefix/withoutPrefix is because we may be 
		// returning eg. a list of items with prefixes 
		// thus modified, not a single PMQIdent

		public T addPrefix(PMQIdent pfx);

		public T rePrefix(PMQIdent newpfx);

//		public T stripPrefix(PMQIdent id);
	}
	// todo - move this





	public abstract class LDBRoot {
		// Root of everything LDB-ish

		public bool Equals([AllowNull] LDBRoot other) {
			throw new LDBShouldNotBeImplementedException(
									"called Equals on LDBRoot");
		}

		public override
			bool Equals([AllowNull] Object other) {
			throw new LDBShouldNotBeImplementedException(
									"called Equals in Object");
		}

		public override int GetHashCode() {
			throw new LDBShouldNotBeImplementedException(
									"called GetHashCode in LDBRoot");
		}

		public static bool operator ==(LDBRoot lhs,
									   LDBRoot rhs) {
			throw new LDBShouldNotBeImplementedException(
									"called == in LDBRoot");
		}

		public static bool operator !=(LDBRoot lhs,
									   LDBRoot rhs) {
			throw new LDBShouldNotBeImplementedException(
									"called != in LDBRoot");
		}


		public virtual string className {
			// todo - this needs checking it works as I want
			// also rename to ldbclassname -- todo
			get => this.GetType().FullName ?? "(unknown)";
		}


		public virtual string classNameQ =>
			"'" + className + "'";


		public virtual string classNamePlus(string plus) =>
			className + "." + plus;

		public override string ToString() {
			// This implicit to-string conversion crap cost me a few hours, so at least
			// do this & blow up at runtime. Would prefer compile time but it seems
			// not possible.  This turns out to have been a good idea,
			// saved me quite a bit more debugging!
			// <https://stackoverflow.com/questions/15345517/is-it-possible-to-disable-implicit-tostring-call>
			// Will try <https://stackoverflow.com/questions/54354342/how-to-get-an-error-shown-when-using-non-strings-as-strings-instead-of-an-autom/54355115#54355115)
			var msg = "LDBRoot.ToString()";
			throw new LDBToStringImplictConversionException(msg);
		}

		public abstract string toDebugStr();
	}



	public abstract class Ephemeral : LDBRoot {
		// for objects that are not part of the AST
	}



	public interface IWithLineNUm {  // todo - use or remove - todo
		//	public 
	}


	public interface IHasTableAlias {
		// Has a table alias AST obj. May be a ...Missing subtype in which
		// it has no alias given by the user, but alias AST is there. More
		// accurately a TableSource alias but ok
		public abstract TableAlias ta { get; set; }
	}


	// Base class for all LDB AST objects
	public abstract class LDBASTobj : LDBRoot {
		// This one is a problem. If I declare the 'parent' property
		// abstract I then have to implement it literally hundreds of
		// times in the derived classes, and I really don't think it
		// would help so just set it to a default which gets overwritten later.
		// 
		// The problem is that those objects were created by the
		// parser, which creates them bottom-up, which means you can't
		// know the parent when they are created. It should be
		// possible to work around this but it does mean reworking the
		// parser fundamentally, and I'm not going to do that
		// assumeing it's even possible.

		// todo - read up on priv/pub/prog accessors I really don't
		// get these at all
		public LDBASTobj parent { get; private set; }
						= noParent;
		// Unique ID per object, useful for distinguishing them
		// without general equality.
		readonly public ulong uid = newUID();

		// public abstract int lineNum { get; set; }

		// Is validated is a cheapish check to ensure everything gets
		// validated
		// can do this as auto prop? todo
		private bool _validated = false;

		protected bool validated {
			get => _validated;
			set {
				// hardMust(!validated, "... already validated... "); - no
				// It's hard to avoid validating a thing twice, so for now allow it
				// Look into it later, really shouldn't happen
				_validated = true;
			}
		}

		public LDBStructsChildren kids { get; private set; } =
					new LDBStructsChildren(noParent); // just easier

		public LDBASTobj() {  //////////////////////////////////////////////////////////////// todo dead?
							  // VStudio thinks this has zero references, which is true explicitly, but it's 
							  // called implicitly on every obj creation - or should do todo- check
			if (recordObjectsCreated) {
				AllCreatedItems.Add(this);
			}
		}


		public virtual void validate() {
			if (reportAlsoIfValidationOK) {
				debugmsg("validating..." + className
					+ " (" + asNameOrIdentContents() + ")");
			}
			mhp_mhk();
			kids.validate();
			// validated = true; -- No, get each subclass to set
			// validated. That forces an explicit call to base() which
			// is more code but ok.
		}


		public bool isValidated { get => validated; }  // todo just => validated;


		public virtual bool isTopLevelItemSeq => false;


		public virtual void mhp_mhk() {  // don't think this need be virtual - todo?
										 // = must have parent, must have kids. The kids may be the
										 // empty set and the parent may be noParent but they must
										 // exist.
										 // Why not just inline this into validate()?
										 // Because a few classes need bespoke validation, but still must check this.
			hardMustParentIsPresent();
			kids.eachKidParentMustBe(this);
		}


		public string asNameOrIdentContents() {
			// for very crude debugging 
			var optName = this is IisNamedItem ni ? ni.name.toRawStr() : "";
			optName = (optName == "" && this is PMQIdent) ? this.toRawStr() : "";

			var optName2 = optName + "  (uid:" + uid.ToString() + ") "
				+ getFirstFewCharsOf(toRawStr());
			return optName2;
		}


		public virtual void getAllUIDs(UIDset uids) {
			uids.Add(this);
			kids.ForEach(namedKid => namedKid.kid.getAllUIDs(uids));
		}


		public bool isStubParent { get => (this is NoParent) || (this is FakeParent); }


		public void validateSpecificItems(params LDBASTobj[] items) {
			Array.ForEach(items, item => item.validate());
		}


		public T? optClimbTo<T>() where T : LDBASTobj {
			var resItem = this;
			while (!(resItem is T)) {
				if (resItem.parent.isStubParent) {
					return null;
				}
				resItem = resItem.parent;
			}

			var res = resItem as T;
			return res;
		}


		public T reqClimbTo<T>() where T : LDBASTobj {
			var loc = className + ".reqClimbTo<T>()";
			var res = optClimbTo<T>();
			hardMust2(!(res is null), 
				// Thanks SO <https://stackoverflow.com/questions/2581642/how-do-i-get-the-type-name-of-a-generic-type-argument>
				// Todo - use this elsewhere, poss ExpectException? todo
				() =>
				loc 
				+ nl + ", climbed to top, didn't find type "
				+ typeof(T).FullName
				+ nl + "Hierarchy is: "
				+ nl + getItemWithParents(this));
			if (res is null) { throw new NNQC(); }
			return res;
		}


		public void hardMustParentIsPresent() {

			hardMust2(!(parent is null), 
				() =>
				 "NULL parent for item of type\n"
				 + $"{GetType()}\n.");

			hardMust2(!(this.parent is NoParent),
				() =>
				"Parent is NoParent on object of type\n"
				+ $"'{this.className}'");
		}


		public void hardMustParentIsAbsent() {
			// For checking cloning - parent should be NoParent immediately 
			// after, before it's set by parent, and never null
			hardMust(!(parent is null),
				 "NULL parent for item of type\n"
				 + $"{GetType()}\n.");
			hardMust((this.parent is NoParent),
				"Parent should be NoParent on object of type\n"
				+ $"'{this.className}'");
		}


		public bool hasParent<P>() {
			hardMust(!(this is NoParent), "Item is NoParent");
			return this.parent is P;
		}


		public bool hasAncestor<A>() {
			hardMust(!this.isStubParent, "Item is NoParent");

			// how to do with a switch expr?
			if (this.parent.isStubParent /* is NoParent || this.parent is FakeParent*/ ) { // todo do as x.hasFudgedParent() - todo
				return false;
			} else if (this.parent is A) {
				return true;
			} else {
				return this.parent.hasAncestor<A>();
			}
		}

		public bool notHasAncestor<A>() =>
			!hasAncestor<A>();


		public void setMyParentTo(LDBASTobj prnt) {
			// An object may only set its parent if its current parent is
			// the dummy 'noParent', which almost every object's parent is
			// set to on creation (see LDBASTobj parent property def).

			// The test for null is peculiar given this. Because
			// noParent is created statically, setting the parent of
			// noParent is difficult, perhaps impossible, to do statically
			// and reliably.Despite the declaration '#nullable enable', it
			// gets created with a null parrent (quite reasonably), hence
			// this check.The actual setting of the parent has to be done
			// at runtime i.e.after the static creation of 'noParent'.

			hardMust2((this.parent is NoParent) || (this.parent is null),
				() =>
				"tried to re-set parent on object of type: "
				+ nl + this.classNameQ
				+ nl + "Original parent type is: "
				+ (this?.parent?.classNameQ == null ? "null" : this.parent.classNameQ)
				+ nl + "and new parent type is: "
				+ prnt.classNameQ
				+ nl + "Item having parent set on it is: "
				// because object may not be fully constructed yet, check howver TODO this is a major hack and must be fixed sometime.
//				+ nl + (this is null ? "(null)" : this.toRawStr()));
				+ nl + this.toRawStr() );

			parent = prnt;
		}


		public void forceNewParentForRootObject() {
			// Every object must have some parent, including the
			// root object of the parse, which is LDBitems
			// (a sequence of LDB items), so this forces one.
			// It is only for that purpose!
			// 
			// LDBitems obj exists for sub objs such as the statements
			// in a while loop, in a proc, in an if... etc., for
			// which their parent is the while/proc/if etc.
			hardMustParentIsAbsent();
			this.parent = fakeParent;
		}


		//public void resetMyParentTo(LDBASTobj prnt) {  ////////////////////////////////// destroy!!!!!! todo
		//	parent = prnt;
		//}

		//////////////////////// xxxxxxxxxxxxx delete - todo
		//private void setParentOnKid(LDBASTobj kid) => // todo needed?
		//											  // check parente doesn't exist - todo
		//	kid.parent = this;

		public virtual void setParentOnKidsAndMakeKidsDict(  // todo - and add to kids dict
															 // this mixes up setting parent and making child dict. Not good. - todo
						params (string kidName, LDBASTobj kid)[] kidsIn) {
			var loc = classNamePlus("setParentOnKidsAndMakeKidsDict");

			var kidsToAdd = new LDBStructsChildren(this, kidsIn);
			hardMust2(kids.isEmpty,
				() =>
				$"setting kids on parent '{className}'" 
				+ "when kids is not empty, "
				+ $"kids is ({kids.Count} kids):"
				+ nl + kids.toDebugStr()
				+ nl + $"Trying to add\n{kidsToAdd.toDebugStr()}\n"
				+ $"in {className}");
			// if there is an LDBStructsChildren and another child,
			// this will blow up - will need to do kids.Add or
			// something - in fact, kids needs to be created in one
			// go, which is what happens.
			kids = kidsToAdd;    ///////////////////////////////////////////////////////// why this? todo
			Array.ForEach(kidsIn, xkid => hardMust(!(xkid.kid is null), loc, "null kid"));
			Array.ForEach(kidsIn, xkid => xkid.kid.setMyParentTo(this));
		}


		public virtual void setParentOnKidsAndAddToKidsDict(
					params (string kidName, LDBASTobj kid)[] kidsIn) {
			var kidsToAdd_justForErrReporting = new LDBStructsChildren(this, kidsIn);
			hardMust2(!kids.isEmpty, // may disable this later 
				() =>
				"adding to kids when kids is empty, "
				 + $"kids is\n{kids.toDebugStr()}\n"
				 + $"Trying to add\n{kidsToAdd_justForErrReporting.toDebugStr()}\n"
				 + $"in {className}");
			Array.ForEach(kidsIn, xkid => {
//				setParentOnKid(xkid.kid);
				xkid.kid.setMyParentTo(this);
				kids.Add(xkid);
			});
		}


		public virtual IdentsUsed getAllIDs() {
		/* This is overridden only in TemplatedAgg, and that's only to 
			deliberately fail. There's something wrong there, todo - 
			un-virtualise this and remove the one override. - todo
		*/
			var iu = new IdentsUsed();
			this._getAllIDs(iu);
			return iu;
		}

		public virtual void _getAllIDs(IdentsUsed iu) {
		// Gets all the identifiers explicitly used (read or written)  ----- todo update this
		// anywhere in the AST.  Does not get idents which are
		// declarations ie. parameters, or var/const declarations
		// unless these are combined with an assignment.
		//
		// Implied idents such as those from tbl.* are implicit and ignored.
		//
		// Root caller supplies new, empty iu
		// need usage here - todo docs//
			kids._getAllIDs(iu);
		}

		public abstract bool canGetAllIdentsUsed {
		// 	This seems to be true on most classes so I could set true 
		// as a default here and override it where it's not wanted, saving 
		// a chunk of code, but I'd rather make it explicit to force me to consider each case. A bit more C&P for less chance of bugs.
			get;
		}


		// Produces a string that should be identical to (excepting
		// whitespaces) the input ie. emit LDB code not valid SQL
		//
		// ToString was causing too many weird problems (due to my
		// inexperience with c#) so just used toRawStr()
		public abstract string toRawStr();

		public override string toDebugStr() => toRawStr();

	} // end LDBASTobj



	public interface IhandlePrefixes<T>
						where T : LDBRoot {

		public bool isPrefixed { get; }

		public bool isExcessPrefixed { get; }

		// The T in addPrefix/rePrefix/withoutPrefix is because we may be 
		// returning eg. a list of items with prefixes 
		// thus modified, not a single PMQIdent

		public T addPrefix(PMQIdent pfx);

		public T rePrefix(PMQIdent newpfx);

//		public T stripPrefix(PMQIdent id);
	}
	// todo - move this





	public abstract class LDBRoot {
		// Root of everything LDB-ish

		public bool Equals([AllowNull] LDBRoot other) {
			throw new LDBShouldNotBeImplementedException(
									"called Equals on LDBRoot");
		}

		public override
			bool Equals([AllowNull] Object other) {
			throw new LDBShouldNotBeImplementedException(
									"called Equals in Object");
		}

		public override int GetHashCode() {
			throw new LDBShouldNotBeImplementedException(
									"called GetHashCode in LDBRoot");
		}

		public static bool operator ==(LDBRoot lhs,
									   LDBRoot rhs) {
			throw new LDBShouldNotBeImplementedException(
									"called == in LDBRoot");
		}

		public static bool operator !=(LDBRoot lhs,
									   LDBRoot rhs) {
			throw new LDBShouldNotBeImplementedException(
									"called != in LDBRoot");
		}


		public virtual string className {
			// todo - this needs checking it works as I want
			// also rename to ldbclassname -- todo
			get => this.GetType().FullName ?? "(unknown)";
		}


		public virtual string classNameQ =>
			"'" + className + "'";


		public virtual string classNamePlus(string plus) =>
			className + "." + plus;

		public override string ToString() {
			// This implicit to-string conversion crap cost me a few hours, so at least
			// do this & blow up at runtime. Would prefer compile time but it seems
			// not possible.  This turns out to have been a good idea,
			// saved me quite a bit more debugging!
			// <https://stackoverflow.com/questions/15345517/is-it-possible-to-disable-implicit-tostring-call>
			// Will try <https://stackoverflow.com/questions/54354342/how-to-get-an-error-shown-when-using-non-strings-as-strings-instead-of-an-autom/54355115#54355115)
			var msg = "LDBRoot.ToString()";
			throw new LDBToStringImplictConversionException(msg);
		}

		public abstract string toDebugStr();
	}



	public abstract class Ephemeral : LDBRoot {
		// for objects that are not part of the AST
	}



	public interface IWithLineNUm {  // todo - use or remove - todo
		//	public 
	}


	public interface IHasTableAlias {
		// Has a table alias AST obj. May be a ...Missing subtype in which
		// it has no alias given by the user, but alias AST is there. More
		// accurately a TableSource alias but ok
		public abstract TableAlias ta { get; set; }
	}


	// Base class for all LDB AST objects
	public abstract class LDBASTobj : LDBRoot {
		// This one is a problem. If I declare the 'parent' property
		// abstract I then have to implement it literally hundreds of
		// times in the derived classes, and I really don't think it
		// would help so just set it to a default which gets overwritten later.
		// 
		// The problem is that those objects were created by the
		// parser, which creates them bottom-up, which means you can't
		// know the parent when they are created. It should be
		// possible to work around this but it does mean reworking the
		// parser fundamentally, and I'm not going to do that
		// assumeing it's even possible.

		// todo - read up on priv/pub/prog accessors I really don't
		// get these at all
		public LDBASTobj parent { get; private set; }
						= noParent;
		// Unique ID per object, useful for distinguishing them
		// without general equality.
		readonly public ulong uid = newUID();

		// public abstract int lineNum { get; set; }

		// Is validated is a cheapish check to ensure everything gets
		// validated
		// can do this as auto prop? todo
		private bool _validated = false;

		protected bool validated {
			get => _validated;
			set {
				// hardMust(!validated, "... already validated... "); - no
				// It's hard to avoid validating a thing twice, so for now allow it
				// Look into it later, really shouldn't happen
				_validated = true;
			}
		}

		public LDBStructsChildren kids { get; private set; } =
					new LDBStructsChildren(noParent); // just easier

		public LDBASTobj() {  //////////////////////////////////////////////////////////////// todo dead?
							  // VStudio thinks this has zero references, which is true explicitly, but it's 
							  // called implicitly on every obj creation - or should do todo- check
			if (recordObjectsCreated) {
				AllCreatedItems.Add(this);
			}
		}


		public virtual void validate() {
			if (reportAlsoIfValidationOK) {
				debugmsg("validating..." + className
					+ " (" + asNameOrIdentContents() + ")");
			}
			mhp_mhk();
			kids.validate();
			// validated = true; -- No, get each subclass to set
			// validated. That forces an explicit call to base() which
			// is more code but ok.
		}


		public bool isValidated { get => validated; }  // todo just => validated;


		public virtual bool isTopLevelItemSeq => false;


		public virtual void mhp_mhk() {  // don't think this need be virtual - todo?
										 // = must have parent, must have kids. The kids may be the
										 // empty set and the parent may be noParent but they must
										 // exist.
										 // Why not just inline this into validate()?
										 // Because a few classes need bespoke validation, but still must check this.
			hardMustParentIsPresent();
			kids.eachKidParentMustBe(this);
		}


		public string asNameOrIdentContents() {
			// for very crude debugging 
			var optName = this is IisNamedItem ni ? ni.name.toRawStr() : "";
			optName = (optName == "" && this is PMQIdent) ? this.toRawStr() : "";

			var optName2 = optName + "  (uid:" + uid.ToString() + ") "
				+ getFirstFewCharsOf(toRawStr());
			return optName2;
		}


		public virtual void getAllUIDs(UIDset uids) {
			uids.Add(this);
			kids.ForEach(namedKid => namedKid.kid.getAllUIDs(uids));
		}


		public bool isStubParent { get => (this is NoParent) || (this is FakeParent); }


		public void validateSpecificItems(params LDBASTobj[] items) {
			Array.ForEach(items, item => item.validate());
		}


		public T? optClimbTo<T>() where T : LDBASTobj {
			var resItem = this;
			while (!(resItem is T)) {
				if (resItem.parent.isStubParent) {
					return null;
				}
				resItem = resItem.parent;
			}

			var res = resItem as T;
			return res;
		}


		public T reqClimbTo<T>() where T : LDBASTobj {
			var loc = className + ".reqClimbTo<T>()";
			var res = optClimbTo<T>();
			hardMust2(!(res is null), 
				// Thanks SO <https://stackoverflow.com/questions/2581642/how-do-i-get-the-type-name-of-a-generic-type-argument>
				// Todo - use this elsewhere, poss ExpectException? todo
				() =>
				loc 
				+ nl + ", climbed to top, didn't find type "
				+ typeof(T).FullName
				+ nl + "Hierarchy is: "
				+ nl + getItemWithParents(this));
			if (res is null) { throw new NNQC(); }
			return res;
		}


		public void hardMustParentIsPresent() {

			hardMust2(!(parent is null), 
				() =>
				 "NULL parent for item of type\n"
				 + $"{GetType()}\n.");

			hardMust2(!(this.parent is NoParent),
				() =>
				"Parent is NoParent on object of type\n"
				+ $"'{this.className}'");
		}


		public void hardMustParentIsAbsent() {
			// For checking cloning - parent should be NoParent immediately 
			// after, before it's set by parent, and never null
			hardMust(!(parent is null),
				 "NULL parent for item of type\n"
				 + $"{GetType()}\n.");
			hardMust((this.parent is NoParent),
				"Parent should be NoParent on object of type\n"
				+ $"'{this.className}'");
		}


		public bool hasParent<P>() {
			hardMust(!(this is NoParent), "Item is NoParent");
			return this.parent is P;
		}


		public bool hasAncestor<A>() {
			hardMust(!this.isStubParent, "Item is NoParent");

			// how to do with a switch expr?
			if (this.parent.isStubParent /* is NoParent || this.parent is FakeParent*/ ) { // todo do as x.hasFudgedParent() - todo
				return false;
			} else if (this.parent is A) {
				return true;
			} else {
				return this.parent.hasAncestor<A>();
			}
		}

		public bool notHasAncestor<A>() =>
			!hasAncestor<A>();


		public void setMyParentTo(LDBASTobj prnt) {
			// An object may only set its parent if its current parent is
			// the dummy 'noParent', which almost every object's parent is
			// set to on creation (see LDBASTobj parent property def).

			// The test for null is peculiar given this. Because
			// noParent is created statically, setting the parent of
			// noParent is difficult, perhaps impossible, to do statically
			// and reliably.Despite the declaration '#nullable enable', it
			// gets created with a null parrent (quite reasonably), hence
			// this check.The actual setting of the parent has to be done
			// at runtime i.e.after the static creation of 'noParent'.

			hardMust2((this.parent is NoParent) || (this.parent is null),
				() =>
				"tried to re-set parent on object of type: "
				+ nl + this.classNameQ
				+ nl + "Original parent type is: "
				+ (this?.parent?.classNameQ == null ? "null" : this.parent.classNameQ)
				+ nl + "and new parent type is: "
				+ prnt.classNameQ
				+ nl + "Item having parent set on it is: "
				// because object may not be fully constructed yet, check howver TODO this is a major hack and must be fixed sometime.
//				+ nl + (this is null ? "(null)" : this.toRawStr()));
				+ nl + this.toRawStr() );

			parent = prnt;
		}


		public void forceNewParentForRootObject() {
			// Every object must have some parent, including the
			// root object of the parse, which is LDBitems
			// (a sequence of LDB items), so this forces one.
			// It is only for that purpose!
			// 
			// LDBitems obj exists for sub objs such as the statements
			// in a while loop, in a proc, in an if... etc., for
			// which their parent is the while/proc/if etc.
			hardMustParentIsAbsent();
			this.parent = fakeParent;
		}


		//public void resetMyParentTo(LDBASTobj prnt) {  ////////////////////////////////// destroy!!!!!! todo
		//	parent = prnt;
		//}

		//////////////////////// xxxxxxxxxxxxx delete - todo
		//private void setParentOnKid(LDBASTobj kid) => // todo needed?
		//											  // check parente doesn't exist - todo
		//	kid.parent = this;

		public virtual void setParentOnKidsAndMakeKidsDict(  // todo - and add to kids dict
															 // this mixes up setting parent and making child dict. Not good. - todo
						params (string kidName, LDBASTobj kid)[] kidsIn) {
			var loc = classNamePlus("setParentOnKidsAndMakeKidsDict");

			var kidsToAdd = new LDBStructsChildren(this, kidsIn);
			hardMust2(kids.isEmpty,
				() =>
				$"setting kids on parent '{className}'" 
				+ "when kids is not empty, "
				+ $"kids is ({kids.Count} kids):"
				+ nl + kids.toDebugStr()
				+ nl + $"Trying to add\n{kidsToAdd.toDebugStr()}\n"
				+ $"in {className}");
			// if there is an LDBStructsChildren and another child,
			// this will blow up - will need to do kids.Add or
			// something - in fact, kids needs to be created in one
			// go, which is what happens.
			kids = kidsToAdd;    ///////////////////////////////////////////////////////// why this? todo
			Array.ForEach(kidsIn, xkid => hardMust(!(xkid.kid is null), loc, "null kid"));
			Array.ForEach(kidsIn, xkid => xkid.kid.setMyParentTo(this));
		}


		public virtual void setParentOnKidsAndAddToKidsDict(
					params (string kidName, LDBASTobj kid)[] kidsIn) {
			var kidsToAdd_justForErrReporting = new LDBStructsChildren(this, kidsIn);
			hardMust2(!kids.isEmpty, // may disable this later 
				() =>
				"adding to kids when kids is empty, "
				 + $"kids is\n{kids.toDebugStr()}\n"
				 + $"Trying to add\n{kidsToAdd_justForErrReporting.toDebugStr()}\n"
				 + $"in {className}");
			Array.ForEach(kidsIn, xkid => {
//				setParentOnKid(xkid.kid);
				xkid.kid.setMyParentTo(this);
				kids.Add(xkid);
			});
		}


		public virtual IdentsUsed getAllIDs() {
		/* This is overridden only in TemplatedAgg, and that's only to 
			deliberately fail. There's something wrong there, todo - 
			un-virtualise this and remove the one override. - todo
		*/
			var iu = new IdentsUsed();
			this._getAllIDs(iu);
			return iu;
		}

		public virtual void _getAllIDs(IdentsUsed iu) {
		// Gets all the identifiers explicitly used (read or written)  ----- todo update this
		// anywhere in the AST.  Does not get idents which are
		// declarations ie. parameters, or var/const declarations
		// unless these are combined with an assignment.
		//
		// Implied idents such as those from tbl.* are implicit and ignored.
		//
		// Root caller supplies new, empty iu
		// need usage here - todo docs//
			kids._getAllIDs(iu);
		}

		public abstract bool canGetAllIdentsUsed {
		// 	This seems to be true on most classes so I could set true 
		// as a default here and override it where it's not wanted, saving 
		// a chunk of code, but I'd rather make it explicit to force me to consider each case. A bit more C&P for less chance of bugs.
			get;
		}


		// Produces a string that should be identical to (excepting
		// whitespaces) the input ie. emit LDB code not valid SQL
		//
		// ToString was causing too many weird problems (due to my
		// inexperience with c#) so just used toRawStr()
		public abstract string toRawStr();

		public override string toDebugStr() => toRawStr();

	} // end LDBASTobj



	public interface IhandlePrefixes<T>
						where T : LDBRoot {

		public bool isPrefixed { get; }

		public bool isExcessPrefixed { get; }

		// The T in addPrefix/rePrefix/withoutPrefix is because we may be 
		// returning eg. a list of items with prefixes 
		// thus modified, not a single PMQIdent

		public T addPrefix(PMQIdent pfx);

		public T rePrefix(PMQIdent newpfx);

//		public T stripPrefix(PMQIdent id);
	}
	// todo - move this





	public abstract class LDBRoot {
		// Root of everything LDB-ish

		public bool Equals([AllowNull] LDBRoot other) {
			throw new LDBShouldNotBeImplementedException(
									"called Equals on LDBRoot");
		}

		public override
			bool Equals([AllowNull] Object other) {
			throw new LDBShouldNotBeImplementedException(
									"called Equals in Object");
		}

		public override int GetHashCode() {
			throw new LDBShouldNotBeImplementedException(
									"called GetHashCode in LDBRoot");
		}

		public static bool operator ==(LDBRoot lhs,
									   LDBRoot rhs) {
			throw new LDBShouldNotBeImplementedException(
									"called == in LDBRoot");
		}

		public static bool operator !=(LDBRoot lhs,
									   LDBRoot rhs) {
			throw new LDBShouldNotBeImplementedException(
									"called != in LDBRoot");
		}


		public virtual string className {
			// todo - this needs checking it works as I want
			// also rename to ldbclassname -- todo
			get => this.GetType().FullName ?? "(unknown)";
		}


		public virtual string classNameQ =>
			"'" + className + "'";


		public virtual string classNamePlus(string plus) =>
			className + "." + plus;

		public override string ToString() {
			// This implicit to-string conversion crap cost me a few hours, so at least
			// do this & blow up at runtime. Would prefer compile time but it seems
			// not possible.  This turns out to have been a good idea,
			// saved me quite a bit more debugging!
			// <https://stackoverflow.com/questions/15345517/is-it-possible-to-disable-implicit-tostring-call>
			// Will try <https://stackoverflow.com/questions/54354342/how-to-get-an-error-shown-when-using-non-strings-as-strings-instead-of-an-autom/54355115#54355115)
			var msg = "LDBRoot.ToString()";
			throw new LDBToStringImplictConversionException(msg);
		}

		public abstract string toDebugStr();
	}



	public abstract class Ephemeral : LDBRoot {
		// for objects that are not part of the AST
	}



	public interface IWithLineNUm {  // todo - use or remove - todo
		//	public 
	}


	public interface IHasTableAlias {
		// Has a table alias AST obj. May be a ...Missing subtype in which
		// it has no alias given by the user, but alias AST is there. More
		// accurately a TableSource alias but ok
		public abstract TableAlias ta { get; set; }
	}


	// Base class for all LDB AST objects
	public abstract class LDBASTobj : LDBRoot {
		// This one is a problem. If I declare the 'parent' property
		// abstract I then have to implement it literally hundreds of
		// times in the derived classes, and I really don't think it
		// would help so just set it to a default which gets overwritten later.
		// 
		// The problem is that those objects were created by the
		// parser, which creates them bottom-up, which means you can't
		// know the parent when they are created. It should be
		// possible to work around this but it does mean reworking the
		// parser fundamentally, and I'm not going to do that
		// assumeing it's even possible.

		// todo - read up on priv/pub/prog accessors I really don't
		// get these at all
		public LDBASTobj parent { get; private set; }
						= noParent;
		// Unique ID per object, useful for distinguishing them
		// without general equality.
		readonly public ulong uid = newUID();

		// public abstract int lineNum { get; set; }

		// Is validated is a cheapish check to ensure everything gets
		// validated
		// can do this as auto prop? todo
		private bool _validated = false;

		protected bool validated {
			get => _validated;
			set {
				// hardMust(!validated, "... already validated... "); - no
				// It's hard to avoid validating a thing twice, so for now allow it
				// Look into it later, really shouldn't happen
				_validated = true;
			}
		}

		public LDBStructsChildren kids { get; private set; } =
					new LDBStructsChildren(noParent); // just easier

		public LDBASTobj() {  //////////////////////////////////////////////////////////////// todo dead?
							  // VStudio thinks this has zero references, which is true explicitly, but it's 
							  // called implicitly on every obj creation - or should do todo- check
			if (recordObjectsCreated) {
				AllCreatedItems.Add(this);
			}
		}


		public virtual void validate() {
			if (reportAlsoIfValidationOK) {
				debugmsg("validating..." + className
					+ " (" + asNameOrIdentContents() + ")");
			}
			mhp_mhk();
			kids.validate();
			// validated = true; -- No, get each subclass to set
			// validated. That forces an explicit call to base() which
			// is more code but ok.
		}


		public bool isValidated { get => validated; }  // todo just => validated;


		public virtual bool isTopLevelItemSeq => false;


		public virtual void mhp_mhk() {  // don't think this need be virtual - todo?
										 // = must have parent, must have kids. The kids may be the
										 // empty set and the parent may be noParent but they must
										 // exist.
										 // Why not just inline this into validate()?
										 // Because a few classes need bespoke validation, but still must check this.
			hardMustParentIsPresent();
			kids.eachKidParentMustBe(this);
		}


		public string asNameOrIdentContents() {
			// for very crude debugging 
			var optName = this is IisNamedItem ni ? ni.name.toRawStr() : "";
			optName = (optName == "" && this is PMQIdent) ? this.toRawStr() : "";

			var optName2 = optName + "  (uid:" + uid.ToString() + ") "
				+ getFirstFewCharsOf(toRawStr());
			return optName2;
		}


		public virtual void getAllUIDs(UIDset uids) {
			uids.Add(this);
			kids.ForEach(namedKid => namedKid.kid.getAllUIDs(uids));
		}


		public bool isStubParent { get => (this is NoParent) || (this is FakeParent); }


		public void validateSpecificItems(params LDBASTobj[] items) {
			Array.ForEach(items, item => item.validate());
		}


		public T? optClimbTo<T>() where T : LDBASTobj {
			var resItem = this;
			while (!(resItem is T)) {
				if (resItem.parent.isStubParent) {
					return null;
				}
				resItem = resItem.parent;
			}

			var res = resItem as T;
			return res;
		}


		public T reqClimbTo<T>() where T : LDBASTobj {
			var loc = className + ".reqClimbTo<T>()";
			var res = optClimbTo<T>();
			hardMust2(!(res is null), 
				// Thanks SO <https://stackoverflow.com/questions/2581642/how-do-i-get-the-type-name-of-a-generic-type-argument>
				// Todo - use this elsewhere, poss ExpectException? todo
				() =>
				loc 
				+ nl + ", climbed to top, didn't find type "
				+ typeof(T).FullName
				+ nl + "Hierarchy is: "
				+ nl + getItemWithParents(this));
			if (res is null) { throw new NNQC(); }
			return res;
		}


		public void hardMustParentIsPresent() {

			hardMust2(!(parent is null), 
				() =>
				 "NULL parent for item of type\n"
				 + $"{GetType()}\n.");

			hardMust2(!(this.parent is NoParent),
				() =>
				"Parent is NoParent on object of type\n"
				+ $"'{this.className}'");
		}


		public void hardMustParentIsAbsent() {
			// For checking cloning - parent should be NoParent immediately 
			// after, before it's set by parent, and never null
			hardMust(!(parent is null),
				 "NULL parent for item of type\n"
				 + $"{GetType()}\n.");
			hardMust((this.parent is NoParent),
				"Parent should be NoParent on object of type\n"
				+ $"'{this.className}'");
		}


		public bool hasParent<P>() {
			hardMust(!(this is NoParent), "Item is NoParent");
			return this.parent is P;
		}


		public bool hasAncestor<A>() {
			hardMust(!this.isStubParent, "Item is NoParent");

			// how to do with a switch expr?
			if (this.parent.isStubParent /* is NoParent || this.parent is FakeParent*/ ) { // todo do as x.hasFudgedParent() - todo
				return false;
			} else if (this.parent is A) {
				return true;
			} else {
				return this.parent.hasAncestor<A>();
			}
		}

		public bool notHasAncestor<A>() =>
			!hasAncestor<A>();


		public void setMyParentTo(LDBASTobj prnt) {
			// An object may only set its parent if its current parent is
			// the dummy 'noParent', which almost every object's parent is
			// set to on creation (see LDBASTobj parent property def).

			// The test for null is peculiar given this. Because
			// noParent is created statically, setting the parent of
			// noParent is difficult, perhaps impossible, to do statically
			// and reliably.Despite the declaration '#nullable enable', it
			// gets created with a null parrent (quite reasonably), hence
			// this check.The actual setting of the parent has to be done
			// at runtime i.e.after the static creation of 'noParent'.

			hardMust2((this.parent is NoParent) || (this.parent is null),
				() =>
				"tried to re-set parent on object of type: "
				+ nl + this.classNameQ
				+ nl + "Original parent type is: "
				+ (this?.parent?.classNameQ == null ? "null" : this.parent.classNameQ)
				+ nl + "and new parent type is: "
				+ prnt.classNameQ
				+ nl + "Item having parent set on it is: "
				// because object may not be fully constructed yet, check howver TODO this is a major hack and must be fixed sometime.
//				+ nl + (this is null ? "(null)" : this.toRawStr()));
				+ nl + this.toRawStr() );

			parent = prnt;
		}


		public void forceNewParentForRootObject() {
			// Every object must have some parent, including the
			// root object of the parse, which is LDBitems
			// (a sequence of LDB items), so this forces one.
			// It is only for that purpose!
			// 
			// LDBitems obj exists for sub objs such as the statements
			// in a while loop, in a proc, in an if... etc., for
			// which their parent is the while/proc/if etc.
			hardMustParentIsAbsent();
			this.parent = fakeParent;
		}


		//public void resetMyParentTo(LDBASTobj prnt) {  ////////////////////////////////// destroy!!!!!! todo
		//	parent = prnt;
		//}

		//////////////////////// xxxxxxxxxxxxx delete - todo
		//private void setParentOnKid(LDBASTobj kid) => // todo needed?
		//											  // check parente doesn't exist - todo
		//	kid.parent = this;

		public virtual void setParentOnKidsAndMakeKidsDict(  // todo - and add to kids dict
															 // this mixes up setting parent and making child dict. Not good. - todo
						params (string kidName, LDBASTobj kid)[] kidsIn) {
			var loc = classNamePlus("setParentOnKidsAndMakeKidsDict");

			var kidsToAdd = new LDBStructsChildren(this, kidsIn);
			hardMust2(kids.isEmpty,
				() =>
				$"setting kids on parent '{className}'" 
				+ "when kids is not empty, "
				+ $"kids is ({kids.Count} kids):"
				+ nl + kids.toDebugStr()
				+ nl + $"Trying to add\n{kidsToAdd.toDebugStr()}\n"
				+ $"in {className}");
			// if there is an LDBStructsChildren and another child,
			// this will blow up - will need to do kids.Add or
			// something - in fact, kids needs to be created in one
			// go, which is what happens.
			kids = kidsToAdd;    ///////////////////////////////////////////////////////// why this? todo
			Array.ForEach(kidsIn, xkid => hardMust(!(xkid.kid is null), loc, "null kid"));
			Array.ForEach(kidsIn, xkid => xkid.kid.setMyParentTo(this));
		}


		public virtual void setParentOnKidsAndAddToKidsDict(
					params (string kidName, LDBASTobj kid)[] kidsIn) {
			var kidsToAdd_justForErrReporting = new LDBStructsChildren(this, kidsIn);
			hardMust2(!kids.isEmpty, // may disable this later 
				() =>
				"adding to kids when kids is empty, "
				 + $"kids is\n{kids.toDebugStr()}\n"
				 + $"Trying to add\n{kidsToAdd_justForErrReporting.toDebugStr()}\n"
				 + $"in {className}");
			Array.ForEach(kidsIn, xkid => {
//				setParentOnKid(xkid.kid);
				xkid.kid.setMyParentTo(this);
				kids.Add(xkid);
			});
		}


		public virtual IdentsUsed getAllIDs() {
		/* This is overridden only in TemplatedAgg, and that's only to 
			deliberately fail. There's something wrong there, todo - 
			un-virtualise this and remove the one override. - todo
		*/
			var iu = new IdentsUsed();
			this._getAllIDs(iu);
			return iu;
		}

		public virtual void _getAllIDs(IdentsUsed iu) {
		// Gets all the identifiers explicitly used (read or written)  ----- todo update this
		// anywhere in the AST.  Does not get idents which are
		// declarations ie. parameters, or var/const declarations
		// unless these are combined with an assignment.
		//
		// Implied idents such as those from tbl.* are implicit and ignored.
		//
		// Root caller supplies new, empty iu
		// need usage here - todo docs//
			kids._getAllIDs(iu);
		}

		public abstract bool canGetAllIdentsUsed {
		// 	This seems to be true on most classes so I could set true 
		// as a default here and override it where it's not wanted, saving 
		// a chunk of code, but I'd rather make it explicit to force me to consider each case. A bit more C&P for less chance of bugs.
			get;
		}


		// Produces a string that should be identical to (excepting
		// whitespaces) the input ie. emit LDB code not valid SQL
		//
		// ToString was causing too many weird problems (due to my
		// inexperience with c#) so just used toRawStr()
		public abstract string toRawStr();

		public override string toDebugStr() => toRawStr();

	} // end LDBASTobj



	public interface IhandlePrefixes<T>
						where T : LDBRoot {

		public bool isPrefixed { get; }

		public bool isExcessPrefixed { get; }

		// The T in addPrefix/rePrefix/withoutPrefix is because we may be 
		// returning eg. a list of items with prefixes 
		// thus modified, not a single PMQIdent

		public T addPrefix(PMQIdent pfx);

		public T rePrefix(PMQIdent newpfx);

//		public T stripPrefix(PMQIdent id);
	}
	// todo - move this





	public abstract class LDBRoot {
		// Root of everything LDB-ish

		public bool Equals([AllowNull] LDBRoot other) {
			throw new LDBShouldNotBeImplementedException(
									"called Equals on LDBRoot");
		}

		public override
			bool Equals([AllowNull] Object other) {
			throw new LDBShouldNotBeImplementedException(
									"called Equals in Object");
		}

		public override int GetHashCode() {
			throw new LDBShouldNotBeImplementedException(
									"called GetHashCode in LDBRoot");
		}

		public static bool operator ==(LDBRoot lhs,
									   LDBRoot rhs) {
			throw new LDBShouldNotBeImplementedException(
									"called == in LDBRoot");
		}

		public static bool operator !=(LDBRoot lhs,
									   LDBRoot rhs) {
			throw new LDBShouldNotBeImplementedException(
									"called != in LDBRoot");
		}


		public virtual string className {
			// todo - this needs checking it works as I want
			// also rename to ldbclassname -- todo
			get => this.GetType().FullName ?? "(unknown)";
		}


		public virtual string classNameQ =>
			"'" + className + "'";


		public virtual string classNamePlus(string plus) =>
			className + "." + plus;

		public override string ToString() {
			// This implicit to-string conversion crap cost me a few hours, so at least
			// do this & blow up at runtime. Would prefer compile time but it seems
			// not possible.  This turns out to have been a good idea,
			// saved me quite a bit more debugging!
			// <https://stackoverflow.com/questions/15345517/is-it-possible-to-disable-implicit-tostring-call>
			// Will try <https://stackoverflow.com/questions/54354342/how-to-get-an-error-shown-when-using-non-strings-as-strings-instead-of-an-autom/54355115#54355115)
			var msg = "LDBRoot.ToString()";
			throw new LDBToStringImplictConversionException(msg);
		}

		public abstract string toDebugStr();
	}



	public abstract class Ephemeral : LDBRoot {
		// for objects that are not part of the AST
	}



	public interface IWithLineNUm {  // todo - use or remove - todo
		//	public 
	}


	public interface IHasTableAlias {
		// Has a table alias AST obj. May be a ...Missing subtype in which
		// it has no alias given by the user, but alias AST is there. More
		// accurately a TableSource alias but ok
		public abstract TableAlias ta { get; set; }
	}


	// Base class for all LDB AST objects
	public abstract class LDBASTobj : LDBRoot {
		// This one is a problem. If I declare the 'parent' property
		// abstract I then have to implement it literally hundreds of
		// times in the derived classes, and I really don't think it
		// would help so just set it to a default which gets overwritten later.
		// 
		// The problem is that those objects were created by the
		// parser, which creates them bottom-up, which means you can't
		// know the parent when they are created. It should be
		// possible to work around this but it does mean reworking the
		// parser fundamentally, and I'm not going to do that
		// assumeing it's even possible.

		// todo - read up on priv/pub/prog accessors I really don't
		// get these at all
		public LDBASTobj parent { get; private set; }
						= noParent;
		// Unique ID per object, useful for distinguishing them
		// without general equality.
		readonly public ulong uid = newUID();

		// public abstract int lineNum { get; set; }

		// Is validated is a cheapish check to ensure everything gets
		// validated
		// can do this as auto prop? todo
		private bool _validated = false;

		protected bool validated {
			get => _validated;
			set {
				// hardMust(!validated, "... already validated... "); - no
				// It's hard to avoid validating a thing twice, so for now allow it
				// Look into it later, really shouldn't happen
				_validated = true;
			}
		}

		public LDBStructsChildren kids { get; private set; } =
					new LDBStructsChildren(noParent); // just easier

		public LDBASTobj() {  //////////////////////////////////////////////////////////////// todo dead?
							  // VStudio thinks this has zero references, which is true explicitly, but it's 
							  // called implicitly on every obj creation - or should do todo- check
			if (recordObjectsCreated) {
				AllCreatedItems.Add(this);
			}
		}


		public virtual void validate() {
			if (reportAlsoIfValidationOK) {
				debugmsg("validating..." + className
					+ " (" + asNameOrIdentContents() + ")");
			}
			mhp_mhk();
			kids.validate();
			// validated = true; -- No, get each subclass to set
			// validated. That forces an explicit call to base() which
			// is more code but ok.
		}


		public bool isValidated { get => validated; }  // todo just => validated;


		public virtual bool isTopLevelItemSeq => false;


		public virtual void mhp_mhk() {  // don't think this need be virtual - todo?
										 // = must have parent, must have kids. The kids may be the
										 // empty set and the parent may be noParent but they must
										 // exist.
										 // Why not just inline this into validate()?
										 // Because a few classes need bespoke validation, but still must check this.
			hardMustParentIsPresent();
			kids.eachKidParentMustBe(this);
		}


		public string asNameOrIdentContents() {
			// for very crude debugging 
			var optName = this is IisNamedItem ni ? ni.name.toRawStr() : "";
			optName = (optName == "" && this is PMQIdent) ? this.toRawStr() : "";

			var optName2 = optName + "  (uid:" + uid.ToString() + ") "
				+ getFirstFewCharsOf(toRawStr());
			return optName2;
		}


		public virtual void getAllUIDs(UIDset uids) {
			uids.Add(this);
			kids.ForEach(namedKid => namedKid.kid.getAllUIDs(uids));
		}


		public bool isStubParent { get => (this is NoParent) || (this is FakeParent); }


		public void validateSpecificItems(params LDBASTobj[] items) {
			Array.ForEach(items, item => item.validate());
		}


		public T? optClimbTo<T>() where T : LDBASTobj {
			var resItem = this;
			while (!(resItem is T)) {
				if (resItem.parent.isStubParent) {
					return null;
				}
				resItem = resItem.parent;
			}

			var res = resItem as T;
			return res;
		}


		public T reqClimbTo<T>() where T : LDBASTobj {
			var loc = className + ".reqClimbTo<T>()";
			var res = optClimbTo<T>();
			hardMust2(!(res is null), 
				// Thanks SO <https://stackoverflow.com/questions/2581642/how-do-i-get-the-type-name-of-a-generic-type-argument>
				// Todo - use this elsewhere, poss ExpectException? todo
				() =>
				loc 
				+ nl + ", climbed to top, didn't find type "
				+ typeof(T).FullName
				+ nl + "Hierarchy is: "
				+ nl + getItemWithParents(this));
			if (res is null) { throw new NNQC(); }
			return res;
		}


		public void hardMustParentIsPresent() {

			hardMust2(!(parent is null), 
				() =>
				 "NULL parent for item of type\n"
				 + $"{GetType()}\n.");

			hardMust2(!(this.parent is NoParent),
				() =>
				"Parent is NoParent on object of type\n"
				+ $"'{this.className}'");
		}


		public void hardMustParentIsAbsent() {
			// For checking cloning - parent should be NoParent immediately 
			// after, before it's set by parent, and never null
			hardMust(!(parent is null),
				 "NULL parent for item of type\n"
				 + $"{GetType()}\n.");
			hardMust((this.parent is NoParent),
				"Parent should be NoParent on object of type\n"
				+ $"'{this.className}'");
		}


		public bool hasParent<P>() {
			hardMust(!(this is NoParent), "Item is NoParent");
			return this.parent is P;
		}


		public bool hasAncestor<A>() {
			hardMust(!this.isStubParent, "Item is NoParent");

			// how to do with a switch expr?
			if (this.parent.isStubParent /* is NoParent || this.parent is FakeParent*/ ) { // todo do as x.hasFudgedParent() - todo
				return false;
			} else if (this.parent is A) {
				return true;
			} else {
				return this.parent.hasAncestor<A>();
			}
		}

		public bool notHasAncestor<A>() =>
			!hasAncestor<A>();


		public void setMyParentTo(LDBASTobj prnt) {
			// An object may only set its parent if its current parent is
			// the dummy 'noParent', which almost every object's parent is
			// set to on creation (see LDBASTobj parent property def).

			// The test for null is peculiar given this. Because
			// noParent is created statically, setting the parent of
			// noParent is difficult, perhaps impossible, to do statically
			// and reliably.Despite the declaration '#nullable enable', it
			// gets created with a null parrent (quite reasonably), hence
			// this check.The actual setting of the parent has to be done
			// at runtime i.e.after the static creation of 'noParent'.

			hardMust2((this.parent is NoParent) || (this.parent is null),
				() =>
				"tried to re-set parent on object of type: "
				+ nl + this.classNameQ
				+ nl + "Original parent type is: "
				+ (this?.parent?.classNameQ == null ? "null" : this.parent.classNameQ)
				+ nl + "and new parent type is: "
				+ prnt.classNameQ
				+ nl + "Item having parent set on it is: "
				// because object may not be fully constructed yet, check howver TODO this is a major hack and must be fixed sometime.
//				+ nl + (this is null ? "(null)" : this.toRawStr()));
				+ nl + this.toRawStr() );

			parent = prnt;
		}


		public void forceNewParentForRootObject() {
			// Every object must have some parent, including the
			// root object of the parse, which is LDBitems
			// (a sequence of LDB items), so this forces one.
			// It is only for that purpose!
			// 
			// LDBitems obj exists for sub objs such as the statements
			// in a while loop, in a proc, in an if... etc., for
			// which their parent is the while/proc/if etc.
			hardMustParentIsAbsent();
			this.parent = fakeParent;
		}


		//public void resetMyParentTo(LDBASTobj prnt) {  ////////////////////////////////// destroy!!!!!! todo
		//	parent = prnt;
		//}

		//////////////////////// xxxxxxxxxxxxx delete - todo
		//private void setParentOnKid(LDBASTobj kid) => // todo needed?
		//											  // check parente doesn't exist - todo
		//	kid.parent = this;

		public virtual void setParentOnKidsAndMakeKidsDict(  // todo - and add to kids dict
															 // this mixes up setting parent and making child dict. Not good. - todo
						params (string kidName, LDBASTobj kid)[] kidsIn) {
			var loc = classNamePlus("setParentOnKidsAndMakeKidsDict");

			var kidsToAdd = new LDBStructsChildren(this, kidsIn);
			hardMust2(kids.isEmpty,
				() =>
				$"setting kids on parent '{className}'" 
				+ "when kids is not empty, "
				+ $"kids is ({kids.Count} kids):"
				+ nl + kids.toDebugStr()
				+ nl + $"Trying to add\n{kidsToAdd.toDebugStr()}\n"
				+ $"in {className}");
			// if there is an LDBStructsChildren and another child,
			// this will blow up - will need to do kids.Add or
			// something - in fact, kids needs to be created in one
			// go, which is what happens.
			kids = kidsToAdd;    ///////////////////////////////////////////////////////// why this? todo
			Array.ForEach(kidsIn, xkid => hardMust(!(xkid.kid is null), loc, "null kid"));
			Array.ForEach(kidsIn, xkid => xkid.kid.setMyParentTo(this));
		}


		public virtual void setParentOnKidsAndAddToKidsDict(
					params (string kidName, LDBASTobj kid)[] kidsIn) {
			var kidsToAdd_justForErrReporting = new LDBStructsChildren(this, kidsIn);
			hardMust2(!kids.isEmpty, // may disable this later 
				() =>
				"adding to kids when kids is empty, "
				 + $"kids is\n{kids.toDebugStr()}\n"
				 + $"Trying to add\n{kidsToAdd_justForErrReporting.toDebugStr()}\n"
				 + $"in {className}");
			Array.ForEach(kidsIn, xkid => {
//				setParentOnKid(xkid.kid);
				xkid.kid.setMyParentTo(this);
				kids.Add(xkid);
			});
		}


		public virtual IdentsUsed getAllIDs() {
		/* This is overridden only in TemplatedAgg, and that's only to 
			deliberately fail. There's something wrong there, todo - 
			un-virtualise this and remove the one override. - todo
		*/
			var iu = new IdentsUsed();
			this._getAllIDs(iu);
			return iu;
		}

		public virtual void _getAllIDs(IdentsUsed iu) {
		// Gets all the identifiers explicitly used (read or written)  ----- todo update this
		// anywhere in the AST.  Does not get idents which are
		// declarations ie. parameters, or var/const declarations
		// unless these are combined with an assignment.
		//
		// Implied idents such as those from tbl.* are implicit and ignored.
		//
		// Root caller supplies new, empty iu
		// need usage here - todo docs//
			kids._getAllIDs(iu);
		}

		public abstract bool canGetAllIdentsUsed {
		// 	This seems to be true on most classes so I could set true 
		// as a default here and override it where it's not wanted, saving 
		// a chunk of code, but I'd rather make it explicit to force me to consider each case. A bit more C&P for less chance of bugs.
			get;
		}


		// Produces a string that should be identical to (excepting
		// whitespaces) the input ie. emit LDB code not valid SQL
		//
		// ToString was causing too many weird problems (due to my
		// inexperience with c#) so just used toRawStr()
		public abstract string toRawStr();

		public override string toDebugStr() => toRawStr();

	} // end LDBASTobj



	public interface IhandlePrefixes<T>
						where T : LDBRoot {

		public bool isPrefixed { get; }

		public bool isExcessPrefixed { get; }

		// The T in addPrefix/rePrefix/withoutPrefix is because we may be 
		// returning eg. a list of items with prefixes 
		// thus modified, not a single PMQIdent

		public T addPrefix(PMQIdent pfx);

		public T rePrefix(PMQIdent newpfx);

//		public T stripPrefix(PMQIdent id);
	}
	// todo - move this





	public abstract class LDBRoot {
		// Root of everything LDB-ish

		public bool Equals([AllowNull] LDBRoot other) {
			throw new LDBShouldNotBeImplementedException(
									"called Equals on LDBRoot");
		}

		public override
			bool Equals([AllowNull] Object other) {
			throw new LDBShouldNotBeImplementedException(
									"called Equals in Object");
		}

		public override int GetHashCode() {
			throw new LDBShouldNotBeImplementedException(
									"called GetHashCode in LDBRoot");
		}

		public static bool operator ==(LDBRoot lhs,
									   LDBRoot rhs) {
			throw new LDBShouldNotBeImplementedException(
									"called == in LDBRoot");
		}

		public static bool operator !=(LDBRoot lhs,
									   LDBRoot rhs) {
			throw new LDBShouldNotBeImplementedException(
									"called != in LDBRoot");
		}


		public virtual string className {
			// todo - this needs checking it works as I want
			// also rename to ldbclassname -- todo
			get => this.GetType().FullName ?? "(unknown)";
		}


		public virtual string classNameQ =>
			"'" + className + "'";


		public virtual string classNamePlus(string plus) =>
			className + "." + plus;

		public override string ToString() {
			// This implicit to-string conversion crap cost me a few hours, so at least
			// do this & blow up at runtime. Would prefer compile time but it seems
			// not possible.  This turns out to have been a good idea,
			// saved me quite a bit more debugging!
			// <https://stackoverflow.com/questions/15345517/is-it-possible-to-disable-implicit-tostring-call>
			// Will try <https://stackoverflow.com/questions/54354342/how-to-get-an-error-shown-when-using-non-strings-as-strings-instead-of-an-autom/54355115#54355115)
			var msg = "LDBRoot.ToString()";
			throw new LDBToStringImplictConversionException(msg);
		}

		public abstract string toDebugStr();
	}



	public abstract class Ephemeral : LDBRoot {
		// for objects that are not part of the AST
	}



	public interface IWithLineNUm {  // todo - use or remove - todo
		//	public 
	}


	public interface IHasTableAlias {
		// Has a table alias AST obj. May be a ...Missing subtype in which
		// it has no alias given by the user, but alias AST is there. More
		// accurately a TableSource alias but ok
		public abstract TableAlias ta { get; set; }
	}


	// Base class for all LDB AST objects
	public abstract class LDBASTobj : LDBRoot {
		// This one is a problem. If I declare the 'parent' property
		// abstract I then have to implement it literally hundreds of
		// times in the derived classes, and I really don't think it
		// would help so just set it to a default which gets overwritten later.
		// 
		// The problem is that those objects were created by the
		// parser, which creates them bottom-up, which means you can't
		// know the parent when they are created. It should be
		// possible to work around this but it does mean reworking the
		// parser fundamentally, and I'm not going to do that
		// assumeing it's even possible.

		// todo - read up on priv/pub/prog accessors I really don't
		// get these at all
		public LDBASTobj parent { get; private set; }
						= noParent;
		// Unique ID per object, useful for distinguishing them
		// without general equality.
		readonly public ulong uid = newUID();

		// public abstract int lineNum { get; set; }

		// Is validated is a cheapish check to ensure everything gets
		// validated
		// can do this as auto prop? todo
		private bool _validated = false;

		protected bool validated {
			get => _validated;
			set {
				// hardMust(!validated, "... already validated... "); - no
				// It's hard to avoid validating a thing twice, so for now allow it
				// Look into it later, really shouldn't happen
				_validated = true;
			}
		}

		public LDBStructsChildren kids { get; private set; } =
					new LDBStructsChildren(noParent); // just easier

		public LDBASTobj() {  //////////////////////////////////////////////////////////////// todo dead?
							  // VStudio thinks this has zero references, which is true explicitly, but it's 
							  // called implicitly on every obj creation - or should do todo- check
			if (recordObjectsCreated) {
				AllCreatedItems.Add(this);
			}
		}


		public virtual void validate() {
			if (reportAlsoIfValidationOK) {
				debugmsg("validating..." + className
					+ " (" + asNameOrIdentContents() + ")");
			}
			mhp_mhk();
			kids.validate();
			// validated = true; -- No, get each subclass to set
			// validated. That forces an explicit call to base() which
			// is more code but ok.
		}


		public bool isValidated { get => validated; }  // todo just => validated;


		public virtual bool isTopLevelItemSeq => false;


		public virtual void mhp_mhk() {  // don't think this need be virtual - todo?
										 // = must have parent, must have kids. The kids may be the
										 // empty set and the parent may be noParent but they must
										 // exist.
										 // Why not just inline this into validate()?
										 // Because a few classes need bespoke validation, but still must check this.
			hardMustParentIsPresent();
			kids.eachKidParentMustBe(this);
		}


		public string asNameOrIdentContents() {
			// for very crude debugging 
			var optName = this is IisNamedItem ni ? ni.name.toRawStr() : "";
			optName = (optName == "" && this is PMQIdent) ? this.toRawStr() : "";

			var optName2 = optName + "  (uid:" + uid.ToString() + ") "
				+ getFirstFewCharsOf(toRawStr());
			return optName2;
		}


		public virtual void getAllUIDs(UIDset uids) {
			uids.Add(this);
			kids.ForEach(namedKid => namedKid.kid.getAllUIDs(uids));
		}


		public bool isStubParent { get => (this is NoParent) || (this is FakeParent); }


		public void validateSpecificItems(params LDBASTobj[] items) {
			Array.ForEach(items, item => item.validate());
		}


		public T? optClimbTo<T>() where T : LDBASTobj {
			var resItem = this;
			while (!(resItem is T)) {
				if (resItem.parent.isStubParent) {
					return null;
				}
				resItem = resItem.parent;
			}

			var res = resItem as T;
			return res;
		}


		public T reqClimbTo<T>() where T : LDBASTobj {
			var loc = className + ".reqClimbTo<T>()";
			var res = optClimbTo<T>();
			hardMust2(!(res is null), 
				// Thanks SO <https://stackoverflow.com/questions/2581642/how-do-i-get-the-type-name-of-a-generic-type-argument>
				// Todo - use this elsewhere, poss ExpectException? todo
				() =>
				loc 
				+ nl + ", climbed to top, didn't find type "
				+ typeof(T).FullName
				+ nl + "Hierarchy is: "
				+ nl + getItemWithParents(this));
			if (res is null) { throw new NNQC(); }
			return res;
		}


		public void hardMustParentIsPresent() {

			hardMust2(!(parent is null), 
				() =>
				 "NULL parent for item of type\n"
				 + $"{GetType()}\n.");

			hardMust2(!(this.parent is NoParent),
				() =>
				"Parent is NoParent on object of type\n"
				+ $"'{this.className}'");
		}


		public void hardMustParentIsAbsent() {
			// For checking cloning - parent should be NoParent immediately 
			// after, before it's set by parent, and never null
			hardMust(!(parent is null),
				 "NULL parent for item of type\n"
				 + $"{GetType()}\n.");
			hardMust((this.parent is NoParent),
				"Parent should be NoParent on object of type\n"
				+ $"'{this.className}'");
		}


		public bool hasParent<P>() {
			hardMust(!(this is NoParent), "Item is NoParent");
			return this.parent is P;
		}


		public bool hasAncestor<A>() {
			hardMust(!this.isStubParent, "Item is NoParent");

			// how to do with a switch expr?
			if (this.parent.isStubParent /* is NoParent || this.parent is FakeParent*/ ) { // todo do as x.hasFudgedParent() - todo
				return false;
			} else if (this.parent is A) {
				return true;
			} else {
				return this.parent.hasAncestor<A>();
			}
		}

		public bool notHasAncestor<A>() =>
			!hasAncestor<A>();


		public void setMyParentTo(LDBASTobj prnt) {
			// An object may only set its parent if its current parent is
			// the dummy 'noParent', which almost every object's parent is
			// set to on creation (see LDBASTobj parent property def).

			// The test for null is peculiar given this. Because
			// noParent is created statically, setting the parent of
			// noParent is difficult, perhaps impossible, to do statically
			// and reliably.Despite the declaration '#nullable enable', it
			// gets created with a null parrent (quite reasonably), hence
			// this check.The actual setting of the parent has to be done
			// at runtime i.e.after the static creation of 'noParent'.

			hardMust2((this.parent is NoParent) || (this.parent is null),
				() =>
				"tried to re-set parent on object of type: "
				+ nl + this.classNameQ
				+ nl + "Original parent type is: "
				+ (this?.parent?.classNameQ == null ? "null" : this.parent.classNameQ)
				+ nl + "and new parent type is: "
				+ prnt.classNameQ
				+ nl + "Item having parent set on it is: "
				// because object may not be fully constructed yet, check howver TODO this is a major hack and must be fixed sometime.
//				+ nl + (this is null ? "(null)" : this.toRawStr()));
				+ nl + this.toRawStr() );

			parent = prnt;
		}


		public void forceNewParentForRootObject() {
			// Every object must have some parent, including the
			// root object of the parse, which is LDBitems
			// (a sequence of LDB items), so this forces one.
			// It is only for that purpose!
			// 
			// LDBitems obj exists for sub objs such as the statements
			// in a while loop, in a proc, in an if... etc., for
			// which their parent is the while/proc/if etc.
			hardMustParentIsAbsent();
			this.parent = fakeParent;
		}


		//public void resetMyParentTo(LDBASTobj prnt) {  ////////////////////////////////// destroy!!!!!! todo
		//	parent = prnt;
		//}

		//////////////////////// xxxxxxxxxxxxx delete - todo
		//private void setParentOnKid(LDBASTobj kid) => // todo needed?
		//											  // check parente doesn't exist - todo
		//	kid.parent = this;

		public virtual void setParentOnKidsAndMakeKidsDict(  // todo - and add to kids dict
															 // this mixes up setting parent and making child dict. Not good. - todo
						params (string kidName, LDBASTobj kid)[] kidsIn) {
			var loc = classNamePlus("setParentOnKidsAndMakeKidsDict");

			var kidsToAdd = new LDBStructsChildren(this, kidsIn);
			hardMust2(kids.isEmpty,
				() =>
				$"setting kids on parent '{className}'" 
				+ "when kids is not empty, "
				+ $"kids is ({kids.Count} kids):"
				+ nl + kids.toDebugStr()
				+ nl + $"Trying to add\n{kidsToAdd.toDebugStr()}\n"
				+ $"in {className}");
			// if there is an LDBStructsChildren and another child,
			// this will blow up - will need to do kids.Add or
			// something - in fact, kids needs to be created in one
			// go, which is what happens.
			kids = kidsToAdd;    ///////////////////////////////////////////////////////// why this? todo
			Array.ForEach(kidsIn, xkid => hardMust(!(xkid.kid is null), loc, "null kid"));
			Array.ForEach(kidsIn, xkid => xkid.kid.setMyParentTo(this));
		}


		public virtual void setParentOnKidsAndAddToKidsDict(
					params (string kidName, LDBASTobj kid)[] kidsIn) {
			var kidsToAdd_justForErrReporting = new LDBStructsChildren(this, kidsIn);
			hardMust2(!kids.isEmpty, // may disable this later 
				() =>
				"adding to kids when kids is empty, "
				 + $"kids is\n{kids.toDebugStr()}\n"
				 + $"Trying to add\n{kidsToAdd_justForErrReporting.toDebugStr()}\n"
				 + $"in {className}");
			Array.ForEach(kidsIn, xkid => {
//				setParentOnKid(xkid.kid);
				xkid.kid.setMyParentTo(this);
				kids.Add(xkid);
			});
		}


		public virtual IdentsUsed getAllIDs() {
		/* This is overridden only in TemplatedAgg, and that's only to 
			deliberately fail. There's something wrong there, todo - 
			un-virtualise this and remove the one override. - todo
		*/
			var iu = new IdentsUsed();
			this._getAllIDs(iu);
			return iu;
		}

		public virtual void _getAllIDs(IdentsUsed iu) {
		// Gets all the identifiers explicitly used (read or written)  ----- todo update this
		// anywhere in the AST.  Does not get idents which are
		// declarations ie. parameters, or var/const declarations
		// unless these are combined with an assignment.
		//
		// Implied idents such as those from tbl.* are implicit and ignored.
		//
		// Root caller supplies new, empty iu
		// need usage here - todo docs//
			kids._getAllIDs(iu);
		}

		public abstract bool canGetAllIdentsUsed {
		// 	This seems to be true on most classes so I could set true 
		// as a default here and override it where it's not wanted, saving 
		// a chunk of code, but I'd rather make it explicit to force me to consider each case. A bit more C&P for less chance of bugs.
			get;
		}


		// Produces a string that should be identical to (excepting
		// whitespaces) the input ie. emit LDB code not valid SQL
		//
		// ToString was causing too many weird problems (due to my
		// inexperience with c#) so just used toRawStr()
		public abstract string toRawStr();

		public override string toDebugStr() => toRawStr();

	} // end LDBASTobj



	public interface IhandlePrefixes<T>
						where T : LDBRoot {

		public bool isPrefixed { get; }

		public bool isExcessPrefixed { get; }

		// The T in addPrefix/rePrefix/withoutPrefix is because we may be 
		// returning eg. a list of items with prefixes 
		// thus modified, not a single PMQIdent

		public T addPrefix(PMQIdent pfx);

		public T rePrefix(PMQIdent newpfx);

//		public T stripPrefix(PMQIdent id);
	}
	// todo - move this





	public abstract class LDBRoot {
		// Root of everything LDB-ish

		public bool Equals([AllowNull] LDBRoot other) {
			throw new LDBShouldNotBeImplementedException(
									"called Equals on LDBRoot");
		}

		public override
			bool Equals([AllowNull] Object other) {
			throw new LDBShouldNotBeImplementedException(
									"called Equals in Object");
		}

		public override int GetHashCode() {
			throw new LDBShouldNotBeImplementedException(
									"called GetHashCode in LDBRoot");
		}

		public static bool operator ==(LDBRoot lhs,
									   LDBRoot rhs) {
			throw new LDBShouldNotBeImplementedException(
									"called == in LDBRoot");
		}

		public static bool operator !=(LDBRoot lhs,
									   LDBRoot rhs) {
			throw new LDBShouldNotBeImplementedException(
									"called != in LDBRoot");
		}


		public virtual string className {
			// todo - this needs checking it works as I want
			// also rename to ldbclassname -- todo
			get => this.GetType().FullName ?? "(unknown)";
		}


		public virtual string classNameQ =>
			"'" + className + "'";


		public virtual string classNamePlus(string plus) =>
			className + "." + plus;

		public override string ToString() {
			// This implicit to-string conversion crap cost me a few hours, so at least
			// do this & blow up at runtime. Would prefer compile time but it seems
			// not possible.  This turns out to have been a good idea,
			// saved me quite a bit more debugging!
			// <https://stackoverflow.com/questions/15345517/is-it-possible-to-disable-implicit-tostring-call>
			// Will try <https://stackoverflow.com/questions/54354342/how-to-get-an-error-shown-when-using-non-strings-as-strings-instead-of-an-autom/54355115#54355115)
			var msg = "LDBRoot.ToString()";
			throw new LDBToStringImplictConversionException(msg);
		}

		public abstract string toDebugStr();
	}



	public abstract class Ephemeral : LDBRoot {
		// for objects that are not part of the AST
	}



	public interface IWithLineNUm {  // todo - use or remove - todo
		//	public 
	}


	public interface IHasTableAlias {
		// Has a table alias AST obj. May be a ...Missing subtype in which
		// it has no alias given by the user, but alias AST is there. More
		// accurately a TableSource alias but ok
		public abstract TableAlias ta { get; set; }
	}


	// Base class for all LDB AST objects
	public abstract class LDBASTobj : LDBRoot {
		// This one is a problem. If I declare the 'parent' property
		// abstract I then have to implement it literally hundreds of
		// times in the derived classes, and I really don't think it
		// would help so just set it to a default which gets overwritten later.
		// 
		// The problem is that those objects were created by the
		// parser, which creates them bottom-up, which means you can't
		// know the parent when they are created. It should be
		// possible to work around this but it does mean reworking the
		// parser fundamentally, and I'm not going to do that
		// assumeing it's even possible.

		// todo - read up on priv/pub/prog accessors I really don't
		// get these at all
		public LDBASTobj parent { get; private set; }
						= noParent;
		// Unique ID per object, useful for distinguishing them
		// without general equality.
		readonly public ulong uid = newUID();

		// public abstract int lineNum { get; set; }

		// Is validated is a cheapish check to ensure everything gets
		// validated
		// can do this as auto prop? todo
		private bool _validated = false;

		protected bool validated {
			get => _validated;
			set {
				// hardMust(!validated, "... already validated... "); - no
				// It's hard to avoid validating a thing twice, so for now allow it
				// Look into it later, really shouldn't happen
				_validated = true;
			}
		}

		public LDBStructsChildren kids { get; private set; } =
					new LDBStructsChildren(noParent); // just easier

		public LDBASTobj() {  //////////////////////////////////////////////////////////////// todo dead?
							  // VStudio thinks this has zero references, which is true explicitly, but it's 
							  // called implicitly on every obj creation - or should do todo- check
			if (recordObjectsCreated) {
				AllCreatedItems.Add(this);
			}
		}


		public virtual void validate() {
			if (reportAlsoIfValidationOK) {
				debugmsg("validating..." + className
					+ " (" + asNameOrIdentContents() + ")");
			}
			mhp_mhk();
			kids.validate();
			// validated = true; -- No, get each subclass to set
			// validated. That forces an explicit call to base() which
			// is more code but ok.
		}


		public bool isValidated { get => validated; }  // todo just => validated;


		public virtual bool isTopLevelItemSeq => false;


		public virtual void mhp_mhk() {  // don't think this need be virtual - todo?
										 // = must have parent, must have kids. The kids may be the
										 // empty set and the parent may be noParent but they must
										 // exist.
										 // Why not just inline this into validate()?
										 // Because a few classes need bespoke validation, but still must check this.
			hardMustParentIsPresent();
			kids.eachKidParentMustBe(this);
		}


		public string asNameOrIdentContents() {
			// for very crude debugging 
			var optName = this is IisNamedItem ni ? ni.name.toRawStr() : "";
			optName = (optName == "" && this is PMQIdent) ? this.toRawStr() : "";

			var optName2 = optName + "  (uid:" + uid.ToString() + ") "
				+ getFirstFewCharsOf(toRawStr());
			return optName2;
		}


		public virtual void getAllUIDs(UIDset uids) {
			uids.Add(this);
			kids.ForEach(namedKid => namedKid.kid.getAllUIDs(uids));
		}


		public bool isStubParent { get => (this is NoParent) || (this is FakeParent); }


		public void validateSpecificItems(params LDBASTobj[] items) {
			Array.ForEach(items, item => item.validate());
		}


		public T? optClimbTo<T>() where T : LDBASTobj {
			var resItem = this;
			while (!(resItem is T)) {
				if (resItem.parent.isStubParent) {
					return null;
				}
				resItem = resItem.parent;
			}

			var res = resItem as T;
			return res;
		}


		public T reqClimbTo<T>() where T : LDBASTobj {
			var loc = className + ".reqClimbTo<T>()";
			var res = optClimbTo<T>();
			hardMust2(!(res is null), 
				// Thanks SO <https://stackoverflow.com/questions/2581642/how-do-i-get-the-type-name-of-a-generic-type-argument>
				// Todo - use this elsewhere, poss ExpectException? todo
				() =>
				loc 
				+ nl + ", climbed to top, didn't find type "
				+ typeof(T).FullName
				+ nl + "Hierarchy is: "
				+ nl + getItemWithParents(this));
			if (res is null) { throw new NNQC(); }
			return res;
		}


		public void hardMustParentIsPresent() {

			hardMust2(!(parent is null), 
				() =>
				 "NULL parent for item of type\n"
				 + $"{GetType()}\n.");

			hardMust2(!(this.parent is NoParent),
				() =>
				"Parent is NoParent on object of type\n"
				+ $"'{this.className}'");
		}


		public void hardMustParentIsAbsent() {
			// For checking cloning - parent should be NoParent immediately 
			// after, before it's set by parent, and never null
			hardMust(!(parent is null),
				 "NULL parent for item of type\n"
				 + $"{GetType()}\n.");
			hardMust((this.parent is NoParent),
				"Parent should be NoParent on object of type\n"
				+ $"'{this.className}'");
		}


		public bool hasParent<P>() {
			hardMust(!(this is NoParent), "Item is NoParent");
			return this.parent is P;
		}


		public bool hasAncestor<A>() {
			hardMust(!this.isStubParent, "Item is NoParent");

			// how to do with a switch expr?
			if (this.parent.isStubParent /* is NoParent || this.parent is FakeParent*/ ) { // todo do as x.hasFudgedParent() - todo
				return false;
			} else if (this.parent is A) {
				return true;
			} else {
				return this.parent.hasAncestor<A>();
			}
		}

		public bool notHasAncestor<A>() =>
			!hasAncestor<A>();


		public void setMyParentTo(LDBASTobj prnt) {
			// An object may only set its parent if its current parent is
			// the dummy 'noParent', which almost every object's parent is
			// set to on creation (see LDBASTobj parent property def).

			// The test for null is peculiar given this. Because
			// noParent is created statically, setting the parent of
			// noParent is difficult, perhaps impossible, to do statically
			// and reliably.Despite the declaration '#nullable enable', it
			// gets created with a null parrent (quite reasonably), hence
			// this check.The actual setting of the parent has to be done
			// at runtime i.e.after the static creation of 'noParent'.

			hardMust2((this.parent is NoParent) || (this.parent is null),
				() =>
				"tried to re-set parent on object of type: "
				+ nl + this.classNameQ
				+ nl + "Original parent type is: "
				+ (this?.parent?.classNameQ == null ? "null" : this.parent.classNameQ)
				+ nl + "and new parent type is: "
				+ prnt.classNameQ
				+ nl + "Item having parent set on it is: "
				// because object may not be fully constructed yet, check howver TODO this is a major hack and must be fixed sometime.
//				+ nl + (this is null ? "(null)" : this.toRawStr()));
				+ nl + this.toRawStr() );

			parent = prnt;
		}


		public void forceNewParentForRootObject() {
			// Every object must have some parent, including the
			// root object of the parse, which is LDBitems
			// (a sequence of LDB items), so this forces one.
			// It is only for that purpose!
			// 
			// LDBitems obj exists for sub objs such as the statements
			// in a while loop, in a proc, in an if... etc., for
			// which their parent is the while/proc/if etc.
			hardMustParentIsAbsent();
			this.parent = fakeParent;
		}


		//public void resetMyParentTo(LDBASTobj prnt) {  ////////////////////////////////// destroy!!!!!! todo
		//	parent = prnt;
		//}

		//////////////////////// xxxxxxxxxxxxx delete - todo
		//private void setParentOnKid(LDBASTobj kid) => // todo needed?
		//											  // check parente doesn't exist - todo
		//	kid.parent = this;

		public virtual void setParentOnKidsAndMakeKidsDict(  // todo - and add to kids dict
															 // this mixes up setting parent and making child dict. Not good. - todo
						params (string kidName, LDBASTobj kid)[] kidsIn) {
			var loc = classNamePlus("setParentOnKidsAndMakeKidsDict");

			var kidsToAdd = new LDBStructsChildren(this, kidsIn);
			hardMust2(kids.isEmpty,
				() =>
				$"setting kids on parent '{className}'" 
				+ "when kids is not empty, "
				+ $"kids is ({kids.Count} kids):"
				+ nl + kids.toDebugStr()
				+ nl + $"Trying to add\n{kidsToAdd.toDebugStr()}\n"
				+ $"in {className}");
			// if there is an LDBStructsChildren and another child,
			// this will blow up - will need to do kids.Add or
			// something - in fact, kids needs to be created in one
			// go, which is what happens.
			kids = kidsToAdd;    ///////////////////////////////////////////////////////// why this? todo
			Array.ForEach(kidsIn, xkid => hardMust(!(xkid.kid is null), loc, "null kid"));
			Array.ForEach(kidsIn, xkid => xkid.kid.setMyParentTo(this));
		}


		public virtual void setParentOnKidsAndAddToKidsDict(
					params (string kidName, LDBASTobj kid)[] kidsIn) {
			var kidsToAdd_justForErrReporting = new LDBStructsChildren(this, kidsIn);
			hardMust2(!kids.isEmpty, // may disable this later 
				() =>
				"adding to kids when kids is empty, "
				 + $"kids is\n{kids.toDebugStr()}\n"
				 + $"Trying to add\n{kidsToAdd_justForErrReporting.toDebugStr()}\n"
				 + $"in {className}");
			Array.ForEach(kidsIn, xkid => {
//				setParentOnKid(xkid.kid);
				xkid.kid.setMyParentTo(this);
				kids.Add(xkid);
			});
		}


		public virtual IdentsUsed getAllIDs() {
		/* This is overridden only in TemplatedAgg, and that's only to 
			deliberately fail. There's something wrong there, todo - 
			un-virtualise this and remove the one override. - todo
		*/
			var iu = new IdentsUsed();
			this._getAllIDs(iu);
			return iu;
		}

		public virtual void _getAllIDs(IdentsUsed iu) {
		// Gets all the identifiers explicitly used (read or written)  ----- todo update this
		// anywhere in the AST.  Does not get idents which are
		// declarations ie. parameters, or var/const declarations
		// unless these are combined with an assignment.
		//
		// Implied idents such as those from tbl.* are implicit and ignored.
		//
		// Root caller supplies new, empty iu
		// need usage here - todo docs//
			kids._getAllIDs(iu);
		}

		public abstract bool canGetAllIdentsUsed {
		// 	This seems to be true on most classes so I could set true 
		// as a default here and override it where it's not wanted, saving 
		// a chunk of code, but I'd rather make it explicit to force me to consider each case. A bit more C&P for less chance of bugs.
			get;
		}


		// Produces a string that should be identical to (excepting
		// whitespaces) the input ie. emit LDB code not valid SQL
		//
		// ToString was causing too many weird problems (due to my
		// inexperience with c#) so just used toRawStr()
		public abstract string toRawStr();

		public override string toDebugStr() => toRawStr();

	} // end LDBASTobj




	public interface IhandlePrefixes<T>
						where T : LDBRoot {

		public bool isPrefixed { get; }

		public bool isExcessPrefixed { get; }

		// The T in addPrefix/rePrefix/withoutPrefix is because we may be 
		// returning eg. a list of items with prefixes 
		// thus modified, not a single PMQIdent

		public T addPrefix(PMQIdent pfx);

		public T rePrefix(PMQIdent newpfx);

//		public T stripPrefix(PMQIdent id);
	}
	// todo - move this





	public abstract class LDBRoot {
		// Root of everything LDB-ish

		public bool Equals([AllowNull] LDBRoot other) {
			throw new LDBShouldNotBeImplementedException(
									"called Equals on LDBRoot");
		}

		public override
			bool Equals([AllowNull] Object other) {
			throw new LDBShouldNotBeImplementedException(
									"called Equals in Object");
		}

		public override int GetHashCode() {
			throw new LDBShouldNotBeImplementedException(
									"called GetHashCode in LDBRoot");
		}

		public static bool operator ==(LDBRoot lhs,
									   LDBRoot rhs) {
			throw new LDBShouldNotBeImplementedException(
									"called == in LDBRoot");
		}

		public static bool operator !=(LDBRoot lhs,
									   LDBRoot rhs) {
			throw new LDBShouldNotBeImplementedException(
									"called != in LDBRoot");
		}


		public virtual string className {
			// todo - this needs checking it works as I want
			// also rename to ldbclassname -- todo
			get => this.GetType().FullName ?? "(unknown)";
		}


		public virtual string classNameQ =>
			"'" + className + "'";


		public virtual string classNamePlus(string plus) =>
			className + "." + plus;

		public override string ToString() {
			// This implicit to-string conversion crap cost me a few hours, so at least
			// do this & blow up at runtime. Would prefer compile time but it seems
			// not possible.  This turns out to have been a good idea,
			// saved me quite a bit more debugging!
			// <https://stackoverflow.com/questions/15345517/is-it-possible-to-disable-implicit-tostring-call>
			// Will try <https://stackoverflow.com/questions/54354342/how-to-get-an-error-shown-when-using-non-strings-as-strings-instead-of-an-autom/54355115#54355115)
			var msg = "LDBRoot.ToString()";
			throw new LDBToStringImplictConversionException(msg);
		}

		public abstract string toDebugStr();
	}



	public abstract class Ephemeral : LDBRoot {
		// for objects that are not part of the AST
	}



	public interface IWithLineNUm {  // todo - use or remove - todo
		//	public 
	}


	public interface IHasTableAlias {
		// Has a table alias AST obj. May be a ...Missing subtype in which
		// it has no alias given by the user, but alias AST is there. More
		// accurately a TableSource alias but ok
		public abstract TableAlias ta { get; set; }
	}


	// Base class for all LDB AST objects
	public abstract class LDBASTobj : LDBRoot {
		// This one is a problem. If I declare the 'parent' property
		// abstract I then have to implement it literally hundreds of
		// times in the derived classes, and I really don't think it
		// would help so just set it to a default which gets overwritten later.
		// 
		// The problem is that those objects were created by the
		// parser, which creates them bottom-up, which means you can't
		// know the parent when they are created. It should be
		// possible to work around this but it does mean reworking the
		// parser fundamentally, and I'm not going to do that
		// assumeing it's even possible.

		// todo - read up on priv/pub/prog accessors I really don't
		// get these at all
		public LDBASTobj parent { get; private set; }
						= noParent;
		// Unique ID per object, useful for distinguishing them
		// without general equality.
		readonly public ulong uid = newUID();

		// public abstract int lineNum { get; set; }

		// Is validated is a cheapish check to ensure everything gets
		// validated
		// can do this as auto prop? todo
		private bool _validated = false;

		protected bool validated {
			get => _validated;
			set {
				// hardMust(!validated, "... already validated... "); - no
				// It's hard to avoid validating a thing twice, so for now allow it
				// Look into it later, really shouldn't happen
				_validated = true;
			}
		}

		public LDBStructsChildren kids { get; private set; } =
					new LDBStructsChildren(noParent); // just easier

		public LDBASTobj() {  //////////////////////////////////////////////////////////////// todo dead?
							  // VStudio thinks this has zero references, which is true explicitly, but it's 
							  // called implicitly on every obj creation - or should do todo- check
			if (recordObjectsCreated) {
				AllCreatedItems.Add(this);
			}
		}


		public virtual void validate() {
			if (reportAlsoIfValidationOK) {
				debugmsg("validating..." + className
					+ " (" + asNameOrIdentContents() + ")");
			}
			mhp_mhk();
			kids.validate();
			// validated = true; -- No, get each subclass to set
			// validated. That forces an explicit call to base() which
			// is more code but ok.
		}


		public bool isValidated { get => validated; }  // todo just => validated;


		public virtual bool isTopLevelItemSeq => false;


		public virtual void mhp_mhk() {  // don't think this need be virtual - todo?
										 // = must have parent, must have kids. The kids may be the
										 // empty set and the parent may be noParent but they must
										 // exist.
										 // Why not just inline this into validate()?
										 // Because a few classes need bespoke validation, but still must check this.
			hardMustParentIsPresent();
			kids.eachKidParentMustBe(this);
		}


		public string asNameOrIdentContents() {
			// for very crude debugging 
			var optName = this is IisNamedItem ni ? ni.name.toRawStr() : "";
			optName = (optName == "" && this is PMQIdent) ? this.toRawStr() : "";

			var optName2 = optName + "  (uid:" + uid.ToString() + ") "
				+ getFirstFewCharsOf(toRawStr());
			return optName2;
		}


		public virtual void getAllUIDs(UIDset uids) {
			uids.Add(this);
			kids.ForEach(namedKid => namedKid.kid.getAllUIDs(uids));
		}


		public bool isStubParent { get => (this is NoParent) || (this is FakeParent); }


		public void validateSpecificItems(params LDBASTobj[] items) {
			Array.ForEach(items, item => item.validate());
		}


		public T? optClimbTo<T>() where T : LDBASTobj {
			var resItem = this;
			while (!(resItem is T)) {
				if (resItem.parent.isStubParent) {
					return null;
				}
				resItem = resItem.parent;
			}

			var res = resItem as T;
			return res;
		}


		public T reqClimbTo<T>() where T : LDBASTobj {
			var loc = className + ".reqClimbTo<T>()";
			var res = optClimbTo<T>();
			hardMust2(!(res is null), 
				// Thanks SO <https://stackoverflow.com/questions/2581642/how-do-i-get-the-type-name-of-a-generic-type-argument>
				// Todo - use this elsewhere, poss ExpectException? todo
				() =>
				loc 
				+ nl + ", climbed to top, didn't find type "
				+ typeof(T).FullName
				+ nl + "Hierarchy is: "
				+ nl + getItemWithParents(this));
			if (res is null) { throw new NNQC(); }
			return res;
		}


		public void hardMustParentIsPresent() {

			hardMust2(!(parent is null), 
				() =>
				 "NULL parent for item of type\n"
				 + $"{GetType()}\n.");

			hardMust2(!(this.parent is NoParent),
				() =>
				"Parent is NoParent on object of type\n"
				+ $"'{this.className}'");
		}


		public void hardMustParentIsAbsent() {
			// For checking cloning - parent should be NoParent immediately 
			// after, before it's set by parent, and never null
			hardMust(!(parent is null),
				 "NULL parent for item of type\n"
				 + $"{GetType()}\n.");
			hardMust((this.parent is NoParent),
				"Parent should be NoParent on object of type\n"
				+ $"'{this.className}'");
		}


		public bool hasParent<P>() {
			hardMust(!(this is NoParent), "Item is NoParent");
			return this.parent is P;
		}


		public bool hasAncestor<A>() {
			hardMust(!this.isStubParent, "Item is NoParent");

			// how to do with a switch expr?
			if (this.parent.isStubParent /* is NoParent || this.parent is FakeParent*/ ) { // todo do as x.hasFudgedParent() - todo
				return false;
			} else if (this.parent is A) {
				return true;
			} else {
				return this.parent.hasAncestor<A>();
			}
		}

		public bool notHasAncestor<A>() =>
			!hasAncestor<A>();


		public void setMyParentTo(LDBASTobj prnt) {
			// An object may only set its parent if its current parent is
			// the dummy 'noParent', which almost every object's parent is
			// set to on creation (see LDBASTobj parent property def).

			// The test for null is peculiar given this. Because
			// noParent is created statically, setting the parent of
			// noParent is difficult, perhaps impossible, to do statically
			// and reliably.Despite the declaration '#nullable enable', it
			// gets created with a null parrent (quite reasonably), hence
			// this check.The actual setting of the parent has to be done
			// at runtime i.e.after the static creation of 'noParent'.

			hardMust2((this.parent is NoParent) || (this.parent is null),
				() =>
				"tried to re-set parent on object of type: "
				+ nl + this.classNameQ
				+ nl + "Original parent type is: "
				+ (this?.parent?.classNameQ == null ? "null" : this.parent.classNameQ)
				+ nl + "and new parent type is: "
				+ prnt.classNameQ
				+ nl + "Item having parent set on it is: "
				// because object may not be fully constructed yet, check howver TODO this is a major hack and must be fixed sometime.
//				+ nl + (this is null ? "(null)" : this.toRawStr()));
				+ nl + this.toRawStr() );

			parent = prnt;
		}


		public void forceNewParentForRootObject() {
			// Every object must have some parent, including the
			// root object of the parse, which is LDBitems
			// (a sequence of LDB items), so this forces one.
			// It is only for that purpose!
			// 
			// LDBitems obj exists for sub objs such as the statements
			// in a while loop, in a proc, in an if... etc., for
			// which their parent is the while/proc/if etc.
			hardMustParentIsAbsent();
			this.parent = fakeParent;
		}


		//public void resetMyParentTo(LDBASTobj prnt) {  ////////////////////////////////// destroy!!!!!! todo
		//	parent = prnt;
		//}

		//////////////////////// xxxxxxxxxxxxx delete - todo
		//private void setParentOnKid(LDBASTobj kid) => // todo needed?
		//											  // check parente doesn't exist - todo
		//	kid.parent = this;

		public virtual void setParentOnKidsAndMakeKidsDict(  // todo - and add to kids dict
															 // this mixes up setting parent and making child dict. Not good. - todo
						params (string kidName, LDBASTobj kid)[] kidsIn) {
			var loc = classNamePlus("setParentOnKidsAndMakeKidsDict");

			var kidsToAdd = new LDBStructsChildren(this, kidsIn);
			hardMust2(kids.isEmpty,
				() =>
				$"setting kids on parent '{className}'" 
				+ "when kids is not empty, "
				+ $"kids is ({kids.Count} kids):"
				+ nl + kids.toDebugStr()
				+ nl + $"Trying to add\n{kidsToAdd.toDebugStr()}\n"
				+ $"in {className}");
			// if there is an LDBStructsChildren and another child,
			// this will blow up - will need to do kids.Add or
			// something - in fact, kids needs to be created in one
			// go, which is what happens.
			kids = kidsToAdd;    ///////////////////////////////////////////////////////// why this? todo
			Array.ForEach(kidsIn, xkid => hardMust(!(xkid.kid is null), loc, "null kid"));
			Array.ForEach(kidsIn, xkid => xkid.kid.setMyParentTo(this));
		}


		public virtual void setParentOnKidsAndAddToKidsDict(
					params (string kidName, LDBASTobj kid)[] kidsIn) {
			var kidsToAdd_justForErrReporting = new LDBStructsChildren(this, kidsIn);
			hardMust2(!kids.isEmpty, // may disable this later 
				() =>
				"adding to kids when kids is empty, "
				 + $"kids is\n{kids.toDebugStr()}\n"
				 + $"Trying to add\n{kidsToAdd_justForErrReporting.toDebugStr()}\n"
				 + $"in {className}");
			Array.ForEach(kidsIn, xkid => {
//				setParentOnKid(xkid.kid);
				xkid.kid.setMyParentTo(this);
				kids.Add(xkid);
			});
		}


		public virtual IdentsUsed getAllIDs() {
		/* This is overridden only in TemplatedAgg, and that's only to 
			deliberately fail. There's something wrong there, todo - 
			un-virtualise this and remove the one override. - todo
		*/
			var iu = new IdentsUsed();
			this._getAllIDs(iu);
			return iu;
		}

		public virtual void _getAllIDs(IdentsUsed iu) {
		// Gets all the identifiers explicitly used (read or written)  ----- todo update this
		// anywhere in the AST.  Does not get idents which are
		// declarations ie. parameters, or var/const declarations
		// unless these are combined with an assignment.
		//
		// Implied idents such as those from tbl.* are implicit and ignored.
		//
		// Root caller supplies new, empty iu
		// need usage here - todo docs//
			kids._getAllIDs(iu);
		}

		public abstract bool canGetAllIdentsUsed {
		// 	This seems to be true on most classes so I could set true 
		// as a default here and override it where it's not wanted, saving 
		// a chunk of code, but I'd rather make it explicit to force me to consider each case. A bit more C&P for less chance of bugs.
			get;
		}


		// Produces a string that should be identical to (excepting
		// whitespaces) the input ie. emit LDB code not valid SQL
		//
		// ToString was causing too many weird problems (due to my
		// inexperience with c#) so just used toRawStr()
		public abstract string toRawStr();

		public override string toDebugStr() => toRawStr();

	} // end LDBASTobj



	public interface IhandlePrefixes<T>
						where T : LDBRoot {

		public bool isPrefixed { get; }

		public bool isExcessPrefixed { get; }

		// The T in addPrefix/rePrefix/withoutPrefix is because we may be 
		// returning eg. a list of items with prefixes 
		// thus modified, not a single PMQIdent

		public T addPrefix(PMQIdent pfx);

		public T rePrefix(PMQIdent newpfx);

//		public T stripPrefix(PMQIdent id);
	}
	// todo - move this





	public abstract class LDBRoot {
		// Root of everything LDB-ish

		public bool Equals([AllowNull] LDBRoot other) {
			throw new LDBShouldNotBeImplementedException(
									"called Equals on LDBRoot");
		}

		public override
			bool Equals([AllowNull] Object other) {
			throw new LDBShouldNotBeImplementedException(
									"called Equals in Object");
		}

		public override int GetHashCode() {
			throw new LDBShouldNotBeImplementedException(
									"called GetHashCode in LDBRoot");
		}

		public static bool operator ==(LDBRoot lhs,
									   LDBRoot rhs) {
			throw new LDBShouldNotBeImplementedException(
									"called == in LDBRoot");
		}

		public static bool operator !=(LDBRoot lhs,
									   LDBRoot rhs) {
			throw new LDBShouldNotBeImplementedException(
									"called != in LDBRoot");
		}


		public virtual string className {
			// todo - this needs checking it works as I want
			// also rename to ldbclassname -- todo
			get => this.GetType().FullName ?? "(unknown)";
		}


		public virtual string classNameQ =>
			"'" + className + "'";


		public virtual string classNamePlus(string plus) =>
			className + "." + plus;

		public override string ToString() {
			// This implicit to-string conversion crap cost me a few hours, so at least
			// do this & blow up at runtime. Would prefer compile time but it seems
			// not possible.  This turns out to have been a good idea,
			// saved me quite a bit more debugging!
			// <https://stackoverflow.com/questions/15345517/is-it-possible-to-disable-implicit-tostring-call>
			// Will try <https://stackoverflow.com/questions/54354342/how-to-get-an-error-shown-when-using-non-strings-as-strings-instead-of-an-autom/54355115#54355115)
			var msg = "LDBRoot.ToString()";
			throw new LDBToStringImplictConversionException(msg);
		}

		public abstract string toDebugStr();
	}



	public abstract class Ephemeral : LDBRoot {
		// for objects that are not part of the AST
	}



	public interface IWithLineNUm {  // todo - use or remove - todo
		//	public 
	}


	public interface IHasTableAlias {
		// Has a table alias AST obj. May be a ...Missing subtype in which
		// it has no alias given by the user, but alias AST is there. More
		// accurately a TableSource alias but ok
		public abstract TableAlias ta { get; set; }
	}


	// Base class for all LDB AST objects
	public abstract class LDBASTobj : LDBRoot {
		// This one is a problem. If I declare the 'parent' property
		// abstract I then have to implement it literally hundreds of
		// times in the derived classes, and I really don't think it
		// would help so just set it to a default which gets overwritten later.
		// 
		// The problem is that those objects were created by the
		// parser, which creates them bottom-up, which means you can't
		// know the parent when they are created. It should be
		// possible to work around this but it does mean reworking the
		// parser fundamentally, and I'm not going to do that
		// assumeing it's even possible.

		// todo - read up on priv/pub/prog accessors I really don't
		// get these at all
		public LDBASTobj parent { get; private set; }
						= noParent;
		// Unique ID per object, useful for distinguishing them
		// without general equality.
		readonly public ulong uid = newUID();

		// public abstract int lineNum { get; set; }

		// Is validated is a cheapish check to ensure everything gets
		// validated
		// can do this as auto prop? todo
		private bool _validated = false;

		protected bool validated {
			get => _validated;
			set {
				// hardMust(!validated, "... already validated... "); - no
				// It's hard to avoid validating a thing twice, so for now allow it
				// Look into it later, really shouldn't happen
				_validated = true;
			}
		}

		public LDBStructsChildren kids { get; private set; } =
					new LDBStructsChildren(noParent); // just easier

		public LDBASTobj() {  //////////////////////////////////////////////////////////////// todo dead?
							  // VStudio thinks this has zero references, which is true explicitly, but it's 
							  // called implicitly on every obj creation - or should do todo- check
			if (recordObjectsCreated) {
				AllCreatedItems.Add(this);
			}
		}


		public virtual void validate() {
			if (reportAlsoIfValidationOK) {
				debugmsg("validating..." + className
					+ " (" + asNameOrIdentContents() + ")");
			}
			mhp_mhk();
			kids.validate();
			// validated = true; -- No, get each subclass to set
			// validated. That forces an explicit call to base() which
			// is more code but ok.
		}


		public bool isValidated { get => validated; }  // todo just => validated;


		public virtual bool isTopLevelItemSeq => false;


		public virtual void mhp_mhk() {  // don't think this need be virtual - todo?
										 // = must have parent, must have kids. The kids may be the
										 // empty set and the parent may be noParent but they must
										 // exist.
										 // Why not just inline this into validate()?
										 // Because a few classes need bespoke validation, but still must check this.
			hardMustParentIsPresent();
			kids.eachKidParentMustBe(this);
		}


		public string asNameOrIdentContents() {
			// for very crude debugging 
			var optName = this is IisNamedItem ni ? ni.name.toRawStr() : "";
			optName = (optName == "" && this is PMQIdent) ? this.toRawStr() : "";

			var optName2 = optName + "  (uid:" + uid.ToString() + ") "
				+ getFirstFewCharsOf(toRawStr());
			return optName2;
		}


		public virtual void getAllUIDs(UIDset uids) {
			uids.Add(this);
			kids.ForEach(namedKid => namedKid.kid.getAllUIDs(uids));
		}


		public bool isStubParent { get => (this is NoParent) || (this is FakeParent); }


		public void validateSpecificItems(params LDBASTobj[] items) {
			Array.ForEach(items, item => item.validate());
		}


		public T? optClimbTo<T>() where T : LDBASTobj {
			var resItem = this;
			while (!(resItem is T)) {
				if (resItem.parent.isStubParent) {
					return null;
				}
				resItem = resItem.parent;
			}

			var res = resItem as T;
			return res;
		}


		public T reqClimbTo<T>() where T : LDBASTobj {
			var loc = className + ".reqClimbTo<T>()";
			var res = optClimbTo<T>();
			hardMust2(!(res is null), 
				// Thanks SO <https://stackoverflow.com/questions/2581642/how-do-i-get-the-type-name-of-a-generic-type-argument>
				// Todo - use this elsewhere, poss ExpectException? todo
				() =>
				loc 
				+ nl + ", climbed to top, didn't find type "
				+ typeof(T).FullName
				+ nl + "Hierarchy is: "
				+ nl + getItemWithParents(this));
			if (res is null) { throw new NNQC(); }
			return res;
		}


		public void hardMustParentIsPresent() {

			hardMust2(!(parent is null), 
				() =>
				 "NULL parent for item of type\n"
				 + $"{GetType()}\n.");

			hardMust2(!(this.parent is NoParent),
				() =>
				"Parent is NoParent on object of type\n"
				+ $"'{this.className}'");
		}


		public void hardMustParentIsAbsent() {
			// For checking cloning - parent should be NoParent immediately 
			// after, before it's set by parent, and never null
			hardMust(!(parent is null),
				 "NULL parent for item of type\n"
				 + $"{GetType()}\n.");
			hardMust((this.parent is NoParent),
				"Parent should be NoParent on object of type\n"
				+ $"'{this.className}'");
		}


		public bool hasParent<P>() {
			hardMust(!(this is NoParent), "Item is NoParent");
			return this.parent is P;
		}


		public bool hasAncestor<A>() {
			hardMust(!this.isStubParent, "Item is NoParent");

			// how to do with a switch expr?
			if (this.parent.isStubParent /* is NoParent || this.parent is FakeParent*/ ) { // todo do as x.hasFudgedParent() - todo
				return false;
			} else if (this.parent is A) {
				return true;
			} else {
				return this.parent.hasAncestor<A>();
			}
		}

		public bool notHasAncestor<A>() =>
			!hasAncestor<A>();


		public void setMyParentTo(LDBASTobj prnt) {
			// An object may only set its parent if its current parent is
			// the dummy 'noParent', which almost every object's parent is
			// set to on creation (see LDBASTobj parent property def).

			// The test for null is peculiar given this. Because
			// noParent is created statically, setting the parent of
			// noParent is difficult, perhaps impossible, to do statically
			// and reliably.Despite the declaration '#nullable enable', it
			// gets created with a null parrent (quite reasonably), hence
			// this check.The actual setting of the parent has to be done
			// at runtime i.e.after the static creation of 'noParent'.

			hardMust2((this.parent is NoParent) || (this.parent is null),
				() =>
				"tried to re-set parent on object of type: "
				+ nl + this.classNameQ
				+ nl + "Original parent type is: "
				+ (this?.parent?.classNameQ == null ? "null" : this.parent.classNameQ)
				+ nl + "and new parent type is: "
				+ prnt.classNameQ
				+ nl + "Item having parent set on it is: "
				// because object may not be fully constructed yet, check howver TODO this is a major hack and must be fixed sometime.
//				+ nl + (this is null ? "(null)" : this.toRawStr()));
				+ nl + this.toRawStr() );

			parent = prnt;
		}


		public void forceNewParentForRootObject() {
			// Every object must have some parent, including the
			// root object of the parse, which is LDBitems
			// (a sequence of LDB items), so this forces one.
			// It is only for that purpose!
			// 
			// LDBitems obj exists for sub objs such as the statements
			// in a while loop, in a proc, in an if... etc., for
			// which their parent is the while/proc/if etc.
			hardMustParentIsAbsent();
			this.parent = fakeParent;
		}


		//public void resetMyParentTo(LDBASTobj prnt) {  ////////////////////////////////// destroy!!!!!! todo
		//	parent = prnt;
		//}

		//////////////////////// xxxxxxxxxxxxx delete - todo
		//private void setParentOnKid(LDBASTobj kid) => // todo needed?
		//											  // check parente doesn't exist - todo
		//	kid.parent = this;

		public virtual void setParentOnKidsAndMakeKidsDict(  // todo - and add to kids dict
															 // this mixes up setting parent and making child dict. Not good. - todo
						params (string kidName, LDBASTobj kid)[] kidsIn) {
			var loc = classNamePlus("setParentOnKidsAndMakeKidsDict");

			var kidsToAdd = new LDBStructsChildren(this, kidsIn);
			hardMust2(kids.isEmpty,
				() =>
				$"setting kids on parent '{className}'" 
				+ "when kids is not empty, "
				+ $"kids is ({kids.Count} kids):"
				+ nl + kids.toDebugStr()
				+ nl + $"Trying to add\n{kidsToAdd.toDebugStr()}\n"
				+ $"in {className}");
			// if there is an LDBStructsChildren and another child,
			// this will blow up - will need to do kids.Add or
			// something - in fact, kids needs to be created in one
			// go, which is what happens.
			kids = kidsToAdd;    ///////////////////////////////////////////////////////// why this? todo
			Array.ForEach(kidsIn, xkid => hardMust(!(xkid.kid is null), loc, "null kid"));
			Array.ForEach(kidsIn, xkid => xkid.kid.setMyParentTo(this));
		}


		public virtual void setParentOnKidsAndAddToKidsDict(
					params (string kidName, LDBASTobj kid)[] kidsIn) {
			var kidsToAdd_justForErrReporting = new LDBStructsChildren(this, kidsIn);
			hardMust2(!kids.isEmpty, // may disable this later 
				() =>
				"adding to kids when kids is empty, "
				 + $"kids is\n{kids.toDebugStr()}\n"
				 + $"Trying to add\n{kidsToAdd_justForErrReporting.toDebugStr()}\n"
				 + $"in {className}");
			Array.ForEach(kidsIn, xkid => {
//				setParentOnKid(xkid.kid);
				xkid.kid.setMyParentTo(this);
				kids.Add(xkid);
			});
		}


		public virtual IdentsUsed getAllIDs() {
		/* This is overridden only in TemplatedAgg, and that's only to 
			deliberately fail. There's something wrong there, todo - 
			un-virtualise this and remove the one override. - todo
		*/
			var iu = new IdentsUsed();
			this._getAllIDs(iu);
			return iu;
		}

		public virtual void _getAllIDs(IdentsUsed iu) {
		// Gets all the identifiers explicitly used (read or written)  ----- todo update this
		// anywhere in the AST.  Does not get idents which are
		// declarations ie. parameters, or var/const declarations
		// unless these are combined with an assignment.
		//
		// Implied idents such as those from tbl.* are implicit and ignored.
		//
		// Root caller supplies new, empty iu
		// need usage here - todo docs//
			kids._getAllIDs(iu);
		}

		public abstract bool canGetAllIdentsUsed {
		// 	This seems to be true on most classes so I could set true 
		// as a default here and override it where it's not wanted, saving 
		// a chunk of code, but I'd rather make it explicit to force me to consider each case. A bit more C&P for less chance of bugs.
			get;
		}


		// Produces a string that should be identical to (excepting
		// whitespaces) the input ie. emit LDB code not valid SQL
		//
		// ToString was causing too many weird problems (due to my
		// inexperience with c#) so just used toRawStr()
		public abstract string toRawStr();

		public override string toDebugStr() => toRawStr();

	} // end LDBASTobj



	public interface IhandlePrefixes<T>
						where T : LDBRoot {

		public bool isPrefixed { get; }

		public bool isExcessPrefixed { get; }

		// The T in addPrefix/rePrefix/withoutPrefix is because we may be 
		// returning eg. a list of items with prefixes 
		// thus modified, not a single PMQIdent

		public T addPrefix(PMQIdent pfx);

		public T rePrefix(PMQIdent newpfx);

//		public T stripPrefix(PMQIdent id);
	}
	// todo - move this





	public abstract class LDBRoot {
		// Root of everything LDB-ish

		public bool Equals([AllowNull] LDBRoot other) {
			throw new LDBShouldNotBeImplementedException(
									"called Equals on LDBRoot");
		}

		public override
			bool Equals([AllowNull] Object other) {
			throw new LDBShouldNotBeImplementedException(
									"called Equals in Object");
		}

		public override int GetHashCode() {
			throw new LDBShouldNotBeImplementedException(
									"called GetHashCode in LDBRoot");
		}

		public static bool operator ==(LDBRoot lhs,
									   LDBRoot rhs) {
			throw new LDBShouldNotBeImplementedException(
									"called == in LDBRoot");
		}

		public static bool operator !=(LDBRoot lhs,
									   LDBRoot rhs) {
			throw new LDBShouldNotBeImplementedException(
									"called != in LDBRoot");
		}


		public virtual string className {
			// todo - this needs checking it works as I want
			// also rename to ldbclassname -- todo
			get => this.GetType().FullName ?? "(unknown)";
		}


		public virtual string classNameQ =>
			"'" + className + "'";


		public virtual string classNamePlus(string plus) =>
			className + "." + plus;

		public override string ToString() {
			// This implicit to-string conversion crap cost me a few hours, so at least
			// do this & blow up at runtime. Would prefer compile time but it seems
			// not possible.  This turns out to have been a good idea,
			// saved me quite a bit more debugging!
			// <https://stackoverflow.com/questions/15345517/is-it-possible-to-disable-implicit-tostring-call>
			// Will try <https://stackoverflow.com/questions/54354342/how-to-get-an-error-shown-when-using-non-strings-as-strings-instead-of-an-autom/54355115#54355115)
			var msg = "LDBRoot.ToString()";
			throw new LDBToStringImplictConversionException(msg);
		}

		public abstract string toDebugStr();
	}



	public abstract class Ephemeral : LDBRoot {
		// for objects that are not part of the AST
	}



	public interface IWithLineNUm {  // todo - use or remove - todo
		//	public 
	}


	public interface IHasTableAlias {
		// Has a table alias AST obj. May be a ...Missing subtype in which
		// it has no alias given by the user, but alias AST is there. More
		// accurately a TableSource alias but ok
		public abstract TableAlias ta { get; set; }
	}


	// Base class for all LDB AST objects
	public abstract class LDBASTobj : LDBRoot {
		// This one is a problem. If I declare the 'parent' property
		// abstract I then have to implement it literally hundreds of
		// times in the derived classes, and I really don't think it
		// would help so just set it to a default which gets overwritten later.
		// 
		// The problem is that those objects were created by the
		// parser, which creates them bottom-up, which means you can't
		// know the parent when they are created. It should be
		// possible to work around this but it does mean reworking the
		// parser fundamentally, and I'm not going to do that
		// assumeing it's even possible.

		// todo - read up on priv/pub/prog accessors I really don't
		// get these at all
		public LDBASTobj parent { get; private set; }
						= noParent;
		// Unique ID per object, useful for distinguishing them
		// without general equality.
		readonly public ulong uid = newUID();

		// public abstract int lineNum { get; set; }

		// Is validated is a cheapish check to ensure everything gets
		// validated
		// can do this as auto prop? todo
		private bool _validated = false;

		protected bool validated {
			get => _validated;
			set {
				// hardMust(!validated, "... already validated... "); - no
				// It's hard to avoid validating a thing twice, so for now allow it
				// Look into it later, really shouldn't happen
				_validated = true;
			}
		}

		public LDBStructsChildren kids { get; private set; } =
					new LDBStructsChildren(noParent); // just easier

		public LDBASTobj() {  //////////////////////////////////////////////////////////////// todo dead?
							  // VStudio thinks this has zero references, which is true explicitly, but it's 
							  // called implicitly on every obj creation - or should do todo- check
			if (recordObjectsCreated) {
				AllCreatedItems.Add(this);
			}
		}


		public virtual void validate() {
			if (reportAlsoIfValidationOK) {
				debugmsg("validating..." + className
					+ " (" + asNameOrIdentContents() + ")");
			}
			mhp_mhk();
			kids.validate();
			// validated = true; -- No, get each subclass to set
			// validated. That forces an explicit call to base() which
			// is more code but ok.
		}


		public bool isValidated { get => validated; }  // todo just => validated;


		public virtual bool isTopLevelItemSeq => false;


		public virtual void mhp_mhk() {  // don't think this need be virtual - todo?
										 // = must have parent, must have kids. The kids may be the
										 // empty set and the parent may be noParent but they must
										 // exist.
										 // Why not just inline this into validate()?
										 // Because a few classes need bespoke validation, but still must check this.
			hardMustParentIsPresent();
			kids.eachKidParentMustBe(this);
		}


		public string asNameOrIdentContents() {
			// for very crude debugging 
			var optName = this is IisNamedItem ni ? ni.name.toRawStr() : "";
			optName = (optName == "" && this is PMQIdent) ? this.toRawStr() : "";

			var optName2 = optName + "  (uid:" + uid.ToString() + ") "
				+ getFirstFewCharsOf(toRawStr());
			return optName2;
		}


		public virtual void getAllUIDs(UIDset uids) {
			uids.Add(this);
			kids.ForEach(namedKid => namedKid.kid.getAllUIDs(uids));
		}


		public bool isStubParent { get => (this is NoParent) || (this is FakeParent); }


		public void validateSpecificItems(params LDBASTobj[] items) {
			Array.ForEach(items, item => item.validate());
		}


		public T? optClimbTo<T>() where T : LDBASTobj {
			var resItem = this;
			while (!(resItem is T)) {
				if (resItem.parent.isStubParent) {
					return null;
				}
				resItem = resItem.parent;
			}

			var res = resItem as T;
			return res;
		}


		public T reqClimbTo<T>() where T : LDBASTobj {
			var loc = className + ".reqClimbTo<T>()";
			var res = optClimbTo<T>();
			hardMust2(!(res is null), 
				// Thanks SO <https://stackoverflow.com/questions/2581642/how-do-i-get-the-type-name-of-a-generic-type-argument>
				// Todo - use this elsewhere, poss ExpectException? todo
				() =>
				loc 
				+ nl + ", climbed to top, didn't find type "
				+ typeof(T).FullName
				+ nl + "Hierarchy is: "
				+ nl + getItemWithParents(this));
			if (res is null) { throw new NNQC(); }
			return res;
		}


		public void hardMustParentIsPresent() {

			hardMust2(!(parent is null), 
				() =>
				 "NULL parent for item of type\n"
				 + $"{GetType()}\n.");

			hardMust2(!(this.parent is NoParent),
				() =>
				"Parent is NoParent on object of type\n"
				+ $"'{this.className}'");
		}


		public void hardMustParentIsAbsent() {
			// For checking cloning - parent should be NoParent immediately 
			// after, before it's set by parent, and never null
			hardMust(!(parent is null),
				 "NULL parent for item of type\n"
				 + $"{GetType()}\n.");
			hardMust((this.parent is NoParent),
				"Parent should be NoParent on object of type\n"
				+ $"'{this.className}'");
		}


		public bool hasParent<P>() {
			hardMust(!(this is NoParent), "Item is NoParent");
			return this.parent is P;
		}


		public bool hasAncestor<A>() {
			hardMust(!this.isStubParent, "Item is NoParent");

			// how to do with a switch expr?
			if (this.parent.isStubParent /* is NoParent || this.parent is FakeParent*/ ) { // todo do as x.hasFudgedParent() - todo
				return false;
			} else if (this.parent is A) {
				return true;
			} else {
				return this.parent.hasAncestor<A>();
			}
		}

		public bool notHasAncestor<A>() =>
			!hasAncestor<A>();


		public void setMyParentTo(LDBASTobj prnt) {
			// An object may only set its parent if its current parent is
			// the dummy 'noParent', which almost every object's parent is
			// set to on creation (see LDBASTobj parent property def).

			// The test for null is peculiar given this. Because
			// noParent is created statically, setting the parent of
			// noParent is difficult, perhaps impossible, to do statically
			// and reliably.Despite the declaration '#nullable enable', it
			// gets created with a null parrent (quite reasonably), hence
			// this check.The actual setting of the parent has to be done
			// at runtime i.e.after the static creation of 'noParent'.

			hardMust2((this.parent is NoParent) || (this.parent is null),
				() =>
				"tried to re-set parent on object of type: "
				+ nl + this.classNameQ
				+ nl + "Original parent type is: "
				+ (this?.parent?.classNameQ == null ? "null" : this.parent.classNameQ)
				+ nl + "and new parent type is: "
				+ prnt.classNameQ
				+ nl + "Item having parent set on it is: "
				// because object may not be fully constructed yet, check howver TODO this is a major hack and must be fixed sometime.
//				+ nl + (this is null ? "(null)" : this.toRawStr()));
				+ nl + this.toRawStr() );

			parent = prnt;
		}


		public void forceNewParentForRootObject() {
			// Every object must have some parent, including the
			// root object of the parse, which is LDBitems
			// (a sequence of LDB items), so this forces one.
			// It is only for that purpose!
			// 
			// LDBitems obj exists for sub objs such as the statements
			// in a while loop, in a proc, in an if... etc., for
			// which their parent is the while/proc/if etc.
			hardMustParentIsAbsent();
			this.parent = fakeParent;
		}


		//public void resetMyParentTo(LDBASTobj prnt) {  ////////////////////////////////// destroy!!!!!! todo
		//	parent = prnt;
		//}

		//////////////////////// xxxxxxxxxxxxx delete - todo
		//private void setParentOnKid(LDBASTobj kid) => // todo needed?
		//											  // check parente doesn't exist - todo
		//	kid.parent = this;

		public virtual void setParentOnKidsAndMakeKidsDict(  // todo - and add to kids dict
															 // this mixes up setting parent and making child dict. Not good. - todo
						params (string kidName, LDBASTobj kid)[] kidsIn) {
			var loc = classNamePlus("setParentOnKidsAndMakeKidsDict");

			var kidsToAdd = new LDBStructsChildren(this, kidsIn);
			hardMust2(kids.isEmpty,
				() =>
				$"setting kids on parent '{className}'" 
				+ "when kids is not empty, "
				+ $"kids is ({kids.Count} kids):"
				+ nl + kids.toDebugStr()
				+ nl + $"Trying to add\n{kidsToAdd.toDebugStr()}\n"
				+ $"in {className}");
			// if there is an LDBStructsChildren and another child,
			// this will blow up - will need to do kids.Add or
			// something - in fact, kids needs to be created in one
			// go, which is what happens.
			kids = kidsToAdd;    ///////////////////////////////////////////////////////// why this? todo
			Array.ForEach(kidsIn, xkid => hardMust(!(xkid.kid is null), loc, "null kid"));
			Array.ForEach(kidsIn, xkid => xkid.kid.setMyParentTo(this));
		}


		public virtual void setParentOnKidsAndAddToKidsDict(
					params (string kidName, LDBASTobj kid)[] kidsIn) {
			var kidsToAdd_justForErrReporting = new LDBStructsChildren(this, kidsIn);
			hardMust2(!kids.isEmpty, // may disable this later 
				() =>
				"adding to kids when kids is empty, "
				 + $"kids is\n{kids.toDebugStr()}\n"
				 + $"Trying to add\n{kidsToAdd_justForErrReporting.toDebugStr()}\n"
				 + $"in {className}");
			Array.ForEach(kidsIn, xkid => {
//				setParentOnKid(xkid.kid);
				xkid.kid.setMyParentTo(this);
				kids.Add(xkid);
			});
		}


		public virtual IdentsUsed getAllIDs() {
		/* This is overridden only in TemplatedAgg, and that's only to 
			deliberately fail. There's something wrong there, todo - 
			un-virtualise this and remove the one override. - todo
		*/
			var iu = new IdentsUsed();
			this._getAllIDs(iu);
			return iu;
		}

		public virtual void _getAllIDs(IdentsUsed iu) {
		// Gets all the identifiers explicitly used (read or written)  ----- todo update this
		// anywhere in the AST.  Does not get idents which are
		// declarations ie. parameters, or var/const declarations
		// unless these are combined with an assignment.
		//
		// Implied idents such as those from tbl.* are implicit and ignored.
		//
		// Root caller supplies new, empty iu
		// need usage here - todo docs//
			kids._getAllIDs(iu);
		}

		public abstract bool canGetAllIdentsUsed {
		// 	This seems to be true on most classes so I could set true 
		// as a default here and override it where it's not wanted, saving 
		// a chunk of code, but I'd rather make it explicit to force me to consider each case. A bit more C&P for less chance of bugs.
			get;
		}


		// Produces a string that should be identical to (excepting
		// whitespaces) the input ie. emit LDB code not valid SQL
		//
		// ToString was causing too many weird problems (due to my
		// inexperience with c#) so just used toRawStr()
		public abstract string toRawStr();

		public override string toDebugStr() => toRawStr();

	} // end LDBASTobj



	public interface IhandlePrefixes<T>
						where T : LDBRoot {

		public bool isPrefixed { get; }

		public bool isExcessPrefixed { get; }

		// The T in addPrefix/rePrefix/withoutPrefix is because we may be 
		// returning eg. a list of items with prefixes 
		// thus modified, not a single PMQIdent

		public T addPrefix(PMQIdent pfx);

		public T rePrefix(PMQIdent newpfx);

//		public T stripPrefix(PMQIdent id);
	}
	// todo - move this





	public abstract class LDBRoot {
		// Root of everything LDB-ish

		public bool Equals([AllowNull] LDBRoot other) {
			throw new LDBShouldNotBeImplementedException(
									"called Equals on LDBRoot");
		}

		public override
			bool Equals([AllowNull] Object other) {
			throw new LDBShouldNotBeImplementedException(
									"called Equals in Object");
		}

		public override int GetHashCode() {
			throw new LDBShouldNotBeImplementedException(
									"called GetHashCode in LDBRoot");
		}

		public static bool operator ==(LDBRoot lhs,
									   LDBRoot rhs) {
			throw new LDBShouldNotBeImplementedException(
									"called == in LDBRoot");
		}

		public static bool operator !=(LDBRoot lhs,
									   LDBRoot rhs) {
			throw new LDBShouldNotBeImplementedException(
									"called != in LDBRoot");
		}


		public virtual string className {
			// todo - this needs checking it works as I want
			// also rename to ldbclassname -- todo
			get => this.GetType().FullName ?? "(unknown)";
		}


		public virtual string classNameQ =>
			"'" + className + "'";


		public virtual string classNamePlus(string plus) =>
			className + "." + plus;

		public override string ToString() {
			// This implicit to-string conversion crap cost me a few hours, so at least
			// do this & blow up at runtime. Would prefer compile time but it seems
			// not possible.  This turns out to have been a good idea,
			// saved me quite a bit more debugging!
			// <https://stackoverflow.com/questions/15345517/is-it-possible-to-disable-implicit-tostring-call>
			// Will try <https://stackoverflow.com/questions/54354342/how-to-get-an-error-shown-when-using-non-strings-as-strings-instead-of-an-autom/54355115#54355115)
			var msg = "LDBRoot.ToString()";
			throw new LDBToStringImplictConversionException(msg);
		}

		public abstract string toDebugStr();
	}



	public abstract class Ephemeral : LDBRoot {
		// for objects that are not part of the AST
	}



	public interface IWithLineNUm {  // todo - use or remove - todo
		//	public 
	}


	public interface IHasTableAlias {
		// Has a table alias AST obj. May be a ...Missing subtype in which
		// it has no alias given by the user, but alias AST is there. More
		// accurately a TableSource alias but ok
		public abstract TableAlias ta { get; set; }
	}


	// Base class for all LDB AST objects
	public abstract class LDBASTobj : LDBRoot {
		// This one is a problem. If I declare the 'parent' property
		// abstract I then have to implement it literally hundreds of
		// times in the derived classes, and I really don't think it
		// would help so just set it to a default which gets overwritten later.
		// 
		// The problem is that those objects were created by the
		// parser, which creates them bottom-up, which means you can't
		// know the parent when they are created. It should be
		// possible to work around this but it does mean reworking the
		// parser fundamentally, and I'm not going to do that
		// assumeing it's even possible.

		// todo - read up on priv/pub/prog accessors I really don't
		// get these at all
		public LDBASTobj parent { get; private set; }
						= noParent;
		// Unique ID per object, useful for distinguishing them
		// without general equality.
		readonly public ulong uid = newUID();

		// public abstract int lineNum { get; set; }

		// Is validated is a cheapish check to ensure everything gets
		// validated
		// can do this as auto prop? todo
		private bool _validated = false;

		protected bool validated {
			get => _validated;
			set {
				// hardMust(!validated, "... already validated... "); - no
				// It's hard to avoid validating a thing twice, so for now allow it
				// Look into it later, really shouldn't happen
				_validated = true;
			}
		}

		public LDBStructsChildren kids { get; private set; } =
					new LDBStructsChildren(noParent); // just easier

		public LDBASTobj() {  //////////////////////////////////////////////////////////////// todo dead?
							  // VStudio thinks this has zero references, which is true explicitly, but it's 
							  // called implicitly on every obj creation - or should do todo- check
			if (recordObjectsCreated) {
				AllCreatedItems.Add(this);
			}
		}


		public virtual void validate() {
			if (reportAlsoIfValidationOK) {
				debugmsg("validating..." + className
					+ " (" + asNameOrIdentContents() + ")");
			}
			mhp_mhk();
			kids.validate();
			// validated = true; -- No, get each subclass to set
			// validated. That forces an explicit call to base() which
			// is more code but ok.
		}


		public bool isValidated { get => validated; }  // todo just => validated;


		public virtual bool isTopLevelItemSeq => false;


		public virtual void mhp_mhk() {  // don't think this need be virtual - todo?
										 // = must have parent, must have kids. The kids may be the
										 // empty set and the parent may be noParent but they must
										 // exist.
										 // Why not just inline this into validate()?
										 // Because a few classes need bespoke validation, but still must check this.
			hardMustParentIsPresent();
			kids.eachKidParentMustBe(this);
		}


		public string asNameOrIdentContents() {
			// for very crude debugging 
			var optName = this is IisNamedItem ni ? ni.name.toRawStr() : "";
			optName = (optName == "" && this is PMQIdent) ? this.toRawStr() : "";

			var optName2 = optName + "  (uid:" + uid.ToString() + ") "
				+ getFirstFewCharsOf(toRawStr());
			return optName2;
		}


		public virtual void getAllUIDs(UIDset uids) {
			uids.Add(this);
			kids.ForEach(namedKid => namedKid.kid.getAllUIDs(uids));
		}


		public bool isStubParent { get => (this is NoParent) || (this is FakeParent); }


		public void validateSpecificItems(params LDBASTobj[] items) {
			Array.ForEach(items, item => item.validate());
		}


		public T? optClimbTo<T>() where T : LDBASTobj {
			var resItem = this;
			while (!(resItem is T)) {
				if (resItem.parent.isStubParent) {
					return null;
				}
				resItem = resItem.parent;
			}

			var res = resItem as T;
			return res;
		}


		public T reqClimbTo<T>() where T : LDBASTobj {
			var loc = className + ".reqClimbTo<T>()";
			var res = optClimbTo<T>();
			hardMust2(!(res is null), 
				// Thanks SO <https://stackoverflow.com/questions/2581642/how-do-i-get-the-type-name-of-a-generic-type-argument>
				// Todo - use this elsewhere, poss ExpectException? todo
				() =>
				loc 
				+ nl + ", climbed to top, didn't find type "
				+ typeof(T).FullName
				+ nl + "Hierarchy is: "
				+ nl + getItemWithParents(this));
			if (res is null) { throw new NNQC(); }
			return res;
		}


		public void hardMustParentIsPresent() {

			hardMust2(!(parent is null), 
				() =>
				 "NULL parent for item of type\n"
				 + $"{GetType()}\n.");

			hardMust2(!(this.parent is NoParent),
				() =>
				"Parent is NoParent on object of type\n"
				+ $"'{this.className}'");
		}


		public void hardMustParentIsAbsent() {
			// For checking cloning - parent should be NoParent immediately 
			// after, before it's set by parent, and never null
			hardMust(!(parent is null),
				 "NULL parent for item of type\n"
				 + $"{GetType()}\n.");
			hardMust((this.parent is NoParent),
				"Parent should be NoParent on object of type\n"
				+ $"'{this.className}'");
		}


		public bool hasParent<P>() {
			hardMust(!(this is NoParent), "Item is NoParent");
			return this.parent is P;
		}


		public bool hasAncestor<A>() {
			hardMust(!this.isStubParent, "Item is NoParent");

			// how to do with a switch expr?
			if (this.parent.isStubParent /* is NoParent || this.parent is FakeParent*/ ) { // todo do as x.hasFudgedParent() - todo
				return false;
			} else if (this.parent is A) {
				return true;
			} else {
				return this.parent.hasAncestor<A>();
			}
		}

		public bool notHasAncestor<A>() =>
			!hasAncestor<A>();


		public void setMyParentTo(LDBASTobj prnt) {
			// An object may only set its parent if its current parent is
			// the dummy 'noParent', which almost every object's parent is
			// set to on creation (see LDBASTobj parent property def).

			// The test for null is peculiar given this. Because
			// noParent is created statically, setting the parent of
			// noParent is difficult, perhaps impossible, to do statically
			// and reliably.Despite the declaration '#nullable enable', it
			// gets created with a null parrent (quite reasonably), hence
			// this check.The actual setting of the parent has to be done
			// at runtime i.e.after the static creation of 'noParent'.

			hardMust2((this.parent is NoParent) || (this.parent is null),
				() =>
				"tried to re-set parent on object of type: "
				+ nl + this.classNameQ
				+ nl + "Original parent type is: "
				+ (this?.parent?.classNameQ == null ? "null" : this.parent.classNameQ)
				+ nl + "and new parent type is: "
				+ prnt.classNameQ
				+ nl + "Item having parent set on it is: "
				// because object may not be fully constructed yet, check howver TODO this is a major hack and must be fixed sometime.
//				+ nl + (this is null ? "(null)" : this.toRawStr()));
				+ nl + this.toRawStr() );

			parent = prnt;
		}


		public void forceNewParentForRootObject() {
			// Every object must have some parent, including the
			// root object of the parse, which is LDBitems
			// (a sequence of LDB items), so this forces one.
			// It is only for that purpose!
			// 
			// LDBitems obj exists for sub objs such as the statements
			// in a while loop, in a proc, in an if... etc., for
			// which their parent is the while/proc/if etc.
			hardMustParentIsAbsent();
			this.parent = fakeParent;
		}


		//public void resetMyParentTo(LDBASTobj prnt) {  ////////////////////////////////// destroy!!!!!! todo
		//	parent = prnt;
		//}

		//////////////////////// xxxxxxxxxxxxx delete - todo
		//private void setParentOnKid(LDBASTobj kid) => // todo needed?
		//											  // check parente doesn't exist - todo
		//	kid.parent = this;

		public virtual void setParentOnKidsAndMakeKidsDict(  // todo - and add to kids dict
															 // this mixes up setting parent and making child dict. Not good. - todo
						params (string kidName, LDBASTobj kid)[] kidsIn) {
			var loc = classNamePlus("setParentOnKidsAndMakeKidsDict");

			var kidsToAdd = new LDBStructsChildren(this, kidsIn);
			hardMust2(kids.isEmpty,
				() =>
				$"setting kids on parent '{className}'" 
				+ "when kids is not empty, "
				+ $"kids is ({kids.Count} kids):"
				+ nl + kids.toDebugStr()
				+ nl + $"Trying to add\n{kidsToAdd.toDebugStr()}\n"
				+ $"in {className}");
			// if there is an LDBStructsChildren and another child,
			// this will blow up - will need to do kids.Add or
			// something - in fact, kids needs to be created in one
			// go, which is what happens.
			kids = kidsToAdd;    ///////////////////////////////////////////////////////// why this? todo
			Array.ForEach(kidsIn, xkid => hardMust(!(xkid.kid is null), loc, "null kid"));
			Array.ForEach(kidsIn, xkid => xkid.kid.setMyParentTo(this));
		}


		public virtual void setParentOnKidsAndAddToKidsDict(
					params (string kidName, LDBASTobj kid)[] kidsIn) {
			var kidsToAdd_justForErrReporting = new LDBStructsChildren(this, kidsIn);
			hardMust2(!kids.isEmpty, // may disable this later 
				() =>
				"adding to kids when kids is empty, "
				 + $"kids is\n{kids.toDebugStr()}\n"
				 + $"Trying to add\n{kidsToAdd_justForErrReporting.toDebugStr()}\n"
				 + $"in {className}");
			Array.ForEach(kidsIn, xkid => {
//				setParentOnKid(xkid.kid);
				xkid.kid.setMyParentTo(this);
				kids.Add(xkid);
			});
		}


		public virtual IdentsUsed getAllIDs() {
		/* This is overridden only in TemplatedAgg, and that's only to 
			deliberately fail. There's something wrong there, todo - 
			un-virtualise this and remove the one override. - todo
		*/
			var iu = new IdentsUsed();
			this._getAllIDs(iu);
			return iu;
		}

		public virtual void _getAllIDs(IdentsUsed iu) {
		// Gets all the identifiers explicitly used (read or written)  ----- todo update this
		// anywhere in the AST.  Does not get idents which are
		// declarations ie. parameters, or var/const declarations
		// unless these are combined with an assignment.
		//
		// Implied idents such as those from tbl.* are implicit and ignored.
		//
		// Root caller supplies new, empty iu
		// need usage here - todo docs//
			kids._getAllIDs(iu);
		}

		public abstract bool canGetAllIdentsUsed {
		// 	This seems to be true on most classes so I could set true 
		// as a default here and override it where it's not wanted, saving 
		// a chunk of code, but I'd rather make it explicit to force me to consider each case. A bit more C&P for less chance of bugs.
			get;
		}


		// Produces a string that should be identical to (excepting
		// whitespaces) the input ie. emit LDB code not valid SQL
		//
		// ToString was causing too many weird problems (due to my
		// inexperience with c#) so just used toRawStr()
		public abstract string toRawStr();

		public override string toDebugStr() => toRawStr();

	} // end LDBASTobj



	public interface IhandlePrefixes<T>
						where T : LDBRoot {

		public bool isPrefixed { get; }

		public bool isExcessPrefixed { get; }

		// The T in addPrefix/rePrefix/withoutPrefix is because we may be 
		// returning eg. a list of items with prefixes 
		// thus modified, not a single PMQIdent

		public T addPrefix(PMQIdent pfx);

		public T rePrefix(PMQIdent newpfx);

//		public T stripPrefix(PMQIdent id);
	}
	// todo - move this





	public abstract class LDBRoot {
		// Root of everything LDB-ish

		public bool Equals([AllowNull] LDBRoot other) {
			throw new LDBShouldNotBeImplementedException(
									"called Equals on LDBRoot");
		}

		public override
			bool Equals([AllowNull] Object other) {
			throw new LDBShouldNotBeImplementedException(
									"called Equals in Object");
		}

		public override int GetHashCode() {
			throw new LDBShouldNotBeImplementedException(
									"called GetHashCode in LDBRoot");
		}

		public static bool operator ==(LDBRoot lhs,
									   LDBRoot rhs) {
			throw new LDBShouldNotBeImplementedException(
									"called == in LDBRoot");
		}

		public static bool operator !=(LDBRoot lhs,
									   LDBRoot rhs) {
			throw new LDBShouldNotBeImplementedException(
									"called != in LDBRoot");
		}


		public virtual string className {
			// todo - this needs checking it works as I want
			// also rename to ldbclassname -- todo
			get => this.GetType().FullName ?? "(unknown)";
		}


		public virtual string classNameQ =>
			"'" + className + "'";


		public virtual string classNamePlus(string plus) =>
			className + "." + plus;

		public override string ToString() {
			// This implicit to-string conversion crap cost me a few hours, so at least
			// do this & blow up at runtime. Would prefer compile time but it seems
			// not possible.  This turns out to have been a good idea,
			// saved me quite a bit more debugging!
			// <https://stackoverflow.com/questions/15345517/is-it-possible-to-disable-implicit-tostring-call>
			// Will try <https://stackoverflow.com/questions/54354342/how-to-get-an-error-shown-when-using-non-strings-as-strings-instead-of-an-autom/54355115#54355115)
			var msg = "LDBRoot.ToString()";
			throw new LDBToStringImplictConversionException(msg);
		}

		public abstract string toDebugStr();
	}



	public abstract class Ephemeral : LDBRoot {
		// for objects that are not part of the AST
	}



	public interface IWithLineNUm {  // todo - use or remove - todo
		//	public 
	}


	public interface IHasTableAlias {
		// Has a table alias AST obj. May be a ...Missing subtype in which
		// it has no alias given by the user, but alias AST is there. More
		// accurately a TableSource alias but ok
		public abstract TableAlias ta { get; set; }
	}


	// Base class for all LDB AST objects
	public abstract class LDBASTobj : LDBRoot {
		// This one is a problem. If I declare the 'parent' property
		// abstract I then have to implement it literally hundreds of
		// times in the derived classes, and I really don't think it
		// would help so just set it to a default which gets overwritten later.
		// 
		// The problem is that those objects were created by the
		// parser, which creates them bottom-up, which means you can't
		// know the parent when they are created. It should be
		// possible to work around this but it does mean reworking the
		// parser fundamentally, and I'm not going to do that
		// assumeing it's even possible.

		// todo - read up on priv/pub/prog accessors I really don't
		// get these at all
		public LDBASTobj parent { get; private set; }
						= noParent;
		// Unique ID per object, useful for distinguishing them
		// without general equality.
		readonly public ulong uid = newUID();

		// public abstract int lineNum { get; set; }

		// Is validated is a cheapish check to ensure everything gets
		// validated
		// can do this as auto prop? todo
		private bool _validated = false;

		protected bool validated {
			get => _validated;
			set {
				// hardMust(!validated, "... already validated... "); - no
				// It's hard to avoid validating a thing twice, so for now allow it
				// Look into it later, really shouldn't happen
				_validated = true;
			}
		}

		public LDBStructsChildren kids { get; private set; } =
					new LDBStructsChildren(noParent); // just easier

		public LDBASTobj() {  //////////////////////////////////////////////////////////////// todo dead?
							  // VStudio thinks this has zero references, which is true explicitly, but it's 
							  // called implicitly on every obj creation - or should do todo- check
			if (recordObjectsCreated) {
				AllCreatedItems.Add(this);
			}
		}


		public virtual void validate() {
			if (reportAlsoIfValidationOK) {
				debugmsg("validating..." + className
					+ " (" + asNameOrIdentContents() + ")");
			}
			mhp_mhk();
			kids.validate();
			// validated = true; -- No, get each subclass to set
			// validated. That forces an explicit call to base() which
			// is more code but ok.
		}


		public bool isValidated { get => validated; }  // todo just => validated;


		public virtual bool isTopLevelItemSeq => false;


		public virtual void mhp_mhk() {  // don't think this need be virtual - todo?
										 // = must have parent, must have kids. The kids may be the
										 // empty set and the parent may be noParent but they must
										 // exist.
										 // Why not just inline this into validate()?
										 // Because a few classes need bespoke validation, but still must check this.
			hardMustParentIsPresent();
			kids.eachKidParentMustBe(this);
		}


		public string asNameOrIdentContents() {
			// for very crude debugging 
			var optName = this is IisNamedItem ni ? ni.name.toRawStr() : "";
			optName = (optName == "" && this is PMQIdent) ? this.toRawStr() : "";

			var optName2 = optName + "  (uid:" + uid.ToString() + ") "
				+ getFirstFewCharsOf(toRawStr());
			return optName2;
		}


		public virtual void getAllUIDs(UIDset uids) {
			uids.Add(this);
			kids.ForEach(namedKid => namedKid.kid.getAllUIDs(uids));
		}


		public bool isStubParent { get => (this is NoParent) || (this is FakeParent); }


		public void validateSpecificItems(params LDBASTobj[] items) {
			Array.ForEach(items, item => item.validate());
		}


		public T? optClimbTo<T>() where T : LDBASTobj {
			var resItem = this;
			while (!(resItem is T)) {
				if (resItem.parent.isStubParent) {
					return null;
				}
				resItem = resItem.parent;
			}

			var res = resItem as T;
			return res;
		}


		public T reqClimbTo<T>() where T : LDBASTobj {
			var loc = className + ".reqClimbTo<T>()";
			var res = optClimbTo<T>();
			hardMust2(!(res is null), 
				// Thanks SO <https://stackoverflow.com/questions/2581642/how-do-i-get-the-type-name-of-a-generic-type-argument>
				// Todo - use this elsewhere, poss ExpectException? todo
				() =>
				loc 
				+ nl + ", climbed to top, didn't find type "
				+ typeof(T).FullName
				+ nl + "Hierarchy is: "
				+ nl + getItemWithParents(this));
			if (res is null) { throw new NNQC(); }
			return res;
		}


		public void hardMustParentIsPresent() {

			hardMust2(!(parent is null), 
				() =>
				 "NULL parent for item of type\n"
				 + $"{GetType()}\n.");

			hardMust2(!(this.parent is NoParent),
				() =>
				"Parent is NoParent on object of type\n"
				+ $"'{this.className}'");
		}


		public void hardMustParentIsAbsent() {
			// For checking cloning - parent should be NoParent immediately 
			// after, before it's set by parent, and never null
			hardMust(!(parent is null),
				 "NULL parent for item of type\n"
				 + $"{GetType()}\n.");
			hardMust((this.parent is NoParent),
				"Parent should be NoParent on object of type\n"
				+ $"'{this.className}'");
		}


		public bool hasParent<P>() {
			hardMust(!(this is NoParent), "Item is NoParent");
			return this.parent is P;
		}


		public bool hasAncestor<A>() {
			hardMust(!this.isStubParent, "Item is NoParent");

			// how to do with a switch expr?
			if (this.parent.isStubParent /* is NoParent || this.parent is FakeParent*/ ) { // todo do as x.hasFudgedParent() - todo
				return false;
			} else if (this.parent is A) {
				return true;
			} else {
				return this.parent.hasAncestor<A>();
			}
		}

		public bool notHasAncestor<A>() =>
			!hasAncestor<A>();


		public void setMyParentTo(LDBASTobj prnt) {
			// An object may only set its parent if its current parent is
			// the dummy 'noParent', which almost every object's parent is
			// set to on creation (see LDBASTobj parent property def).

			// The test for null is peculiar given this. Because
			// noParent is created statically, setting the parent of
			// noParent is difficult, perhaps impossible, to do statically
			// and reliably.Despite the declaration '#nullable enable', it
			// gets created with a null parrent (quite reasonably), hence
			// this check.The actual setting of the parent has to be done
			// at runtime i.e.after the static creation of 'noParent'.

			hardMust2((this.parent is NoParent) || (this.parent is null),
				() =>
				"tried to re-set parent on object of type: "
				+ nl + this.classNameQ
				+ nl + "Original parent type is: "
				+ (this?.parent?.classNameQ == null ? "null" : this.parent.classNameQ)
				+ nl + "and new parent type is: "
				+ prnt.classNameQ
				+ nl + "Item having parent set on it is: "
				// because object may not be fully constructed yet, check howver TODO this is a major hack and must be fixed sometime.
//				+ nl + (this is null ? "(null)" : this.toRawStr()));
				+ nl + this.toRawStr() );

			parent = prnt;
		}


		public void forceNewParentForRootObject() {
			// Every object must have some parent, including the
			// root object of the parse, which is LDBitems
			// (a sequence of LDB items), so this forces one.
			// It is only for that purpose!
			// 
			// LDBitems obj exists for sub objs such as the statements
			// in a while loop, in a proc, in an if... etc., for
			// which their parent is the while/proc/if etc.
			hardMustParentIsAbsent();
			this.parent = fakeParent;
		}


		//public void resetMyParentTo(LDBASTobj prnt) {  ////////////////////////////////// destroy!!!!!! todo
		//	parent = prnt;
		//}

		//////////////////////// xxxxxxxxxxxxx delete - todo
		//private void setParentOnKid(LDBASTobj kid) => // todo needed?
		//											  // check parente doesn't exist - todo
		//	kid.parent = this;

		public virtual void setParentOnKidsAndMakeKidsDict(  // todo - and add to kids dict
															 // this mixes up setting parent and making child dict. Not good. - todo
						params (string kidName, LDBASTobj kid)[] kidsIn) {
			var loc = classNamePlus("setParentOnKidsAndMakeKidsDict");

			var kidsToAdd = new LDBStructsChildren(this, kidsIn);
			hardMust2(kids.isEmpty,
				() =>
				$"setting kids on parent '{className}'" 
				+ "when kids is not empty, "
				+ $"kids is ({kids.Count} kids):"
				+ nl + kids.toDebugStr()
				+ nl + $"Trying to add\n{kidsToAdd.toDebugStr()}\n"
				+ $"in {className}");
			// if there is an LDBStructsChildren and another child,
			// this will blow up - will need to do kids.Add or
			// something - in fact, kids needs to be created in one
			// go, which is what happens.
			kids = kidsToAdd;    ///////////////////////////////////////////////////////// why this? todo
			Array.ForEach(kidsIn, xkid => hardMust(!(xkid.kid is null), loc, "null kid"));
			Array.ForEach(kidsIn, xkid => xkid.kid.setMyParentTo(this));
		}


		public virtual void setParentOnKidsAndAddToKidsDict(
					params (string kidName, LDBASTobj kid)[] kidsIn) {
			var kidsToAdd_justForErrReporting = new LDBStructsChildren(this, kidsIn);
			hardMust2(!kids.isEmpty, // may disable this later 
				() =>
				"adding to kids when kids is empty, "
				 + $"kids is\n{kids.toDebugStr()}\n"
				 + $"Trying to add\n{kidsToAdd_justForErrReporting.toDebugStr()}\n"
				 + $"in {className}");
			Array.ForEach(kidsIn, xkid => {
//				setParentOnKid(xkid.kid);
				xkid.kid.setMyParentTo(this);
				kids.Add(xkid);
			});
		}


		public virtual IdentsUsed getAllIDs() {
		/* This is overridden only in TemplatedAgg, and that's only to 
			deliberately fail. There's something wrong there, todo - 
			un-virtualise this and remove the one override. - todo
		*/
			var iu = new IdentsUsed();
			this._getAllIDs(iu);
			return iu;
		}

		public virtual void _getAllIDs(IdentsUsed iu) {
		// Gets all the identifiers explicitly used (read or written)  ----- todo update this
		// anywhere in the AST.  Does not get idents which are
		// declarations ie. parameters, or var/const declarations
		// unless these are combined with an assignment.
		//
		// Implied idents such as those from tbl.* are implicit and ignored.
		//
		// Root caller supplies new, empty iu
		// need usage here - todo docs//
			kids._getAllIDs(iu);
		}

		public abstract bool canGetAllIdentsUsed {
		// 	This seems to be true on most classes so I could set true 
		// as a default here and override it where it's not wanted, saving 
		// a chunk of code, but I'd rather make it explicit to force me to consider each case. A bit more C&P for less chance of bugs.
			get;
		}


		// Produces a string that should be identical to (excepting
		// whitespaces) the input ie. emit LDB code not valid SQL
		//
		// ToString was causing too many weird problems (due to my
		// inexperience with c#) so just used toRawStr()
		public abstract string toRawStr();

		public override string toDebugStr() => toRawStr();

	} // end LDBASTobj



	public interface IhandlePrefixes<T>
						where T : LDBRoot {

		public bool isPrefixed { get; }

		public bool isExcessPrefixed { get; }

		// The T in addPrefix/rePrefix/withoutPrefix is because we may be 
		// returning eg. a list of items with prefixes 
		// thus modified, not a single PMQIdent

		public T addPrefix(PMQIdent pfx);

		public T rePrefix(PMQIdent newpfx);

//		public T stripPrefix(PMQIdent id);
	}
	// todo - move this





	public abstract class LDBRoot {
		// Root of everything LDB-ish

		public bool Equals([AllowNull] LDBRoot other) {
			throw new LDBShouldNotBeImplementedException(
									"called Equals on LDBRoot");
		}

		public override
			bool Equals([AllowNull] Object other) {
			throw new LDBShouldNotBeImplementedException(
									"called Equals in Object");
		}

		public override int GetHashCode() {
			throw new LDBShouldNotBeImplementedException(
									"called GetHashCode in LDBRoot");
		}

		public static bool operator ==(LDBRoot lhs,
									   LDBRoot rhs) {
			throw new LDBShouldNotBeImplementedException(
									"called == in LDBRoot");
		}

		public static bool operator !=(LDBRoot lhs,
									   LDBRoot rhs) {
			throw new LDBShouldNotBeImplementedException(
									"called != in LDBRoot");
		}


		public virtual string className {
			// todo - this needs checking it works as I want
			// also rename to ldbclassname -- todo
			get => this.GetType().FullName ?? "(unknown)";
		}


		public virtual string classNameQ =>
			"'" + className + "'";


		public virtual string classNamePlus(string plus) =>
			className + "." + plus;

		public override string ToString() {
			// This implicit to-string conversion crap cost me a few hours, so at least
			// do this & blow up at runtime. Would prefer compile time but it seems
			// not possible.  This turns out to have been a good idea,
			// saved me quite a bit more debugging!
			// <https://stackoverflow.com/questions/15345517/is-it-possible-to-disable-implicit-tostring-call>
			// Will try <https://stackoverflow.com/questions/54354342/how-to-get-an-error-shown-when-using-non-strings-as-strings-instead-of-an-autom/54355115#54355115)
			var msg = "LDBRoot.ToString()";
			throw new LDBToStringImplictConversionException(msg);
		}

		public abstract string toDebugStr();
	}



	public abstract class Ephemeral : LDBRoot {
		// for objects that are not part of the AST
	}



	public interface IWithLineNUm {  // todo - use or remove - todo
		//	public 
	}


	public interface IHasTableAlias {
		// Has a table alias AST obj. May be a ...Missing subtype in which
		// it has no alias given by the user, but alias AST is there. More
		// accurately a TableSource alias but ok
		public abstract TableAlias ta { get; set; }
	}


	// Base class for all LDB AST objects
	public abstract class LDBASTobj : LDBRoot {
		// This one is a problem. If I declare the 'parent' property
		// abstract I then have to implement it literally hundreds of
		// times in the derived classes, and I really don't think it
		// would help so just set it to a default which gets overwritten later.
		// 
		// The problem is that those objects were created by the
		// parser, which creates them bottom-up, which means you can't
		// know the parent when they are created. It should be
		// possible to work around this but it does mean reworking the
		// parser fundamentally, and I'm not going to do that
		// assumeing it's even possible.

		// todo - read up on priv/pub/prog accessors I really don't
		// get these at all
		public LDBASTobj parent { get; private set; }
						= noParent;
		// Unique ID per object, useful for distinguishing them
		// without general equality.
		readonly public ulong uid = newUID();

		// public abstract int lineNum { get; set; }

		// Is validated is a cheapish check to ensure everything gets
		// validated
		// can do this as auto prop? todo
		private bool _validated = false;

		protected bool validated {
			get => _validated;
			set {
				// hardMust(!validated, "... already validated... "); - no
				// It's hard to avoid validating a thing twice, so for now allow it
				// Look into it later, really shouldn't happen
				_validated = true;
			}
		}

		public LDBStructsChildren kids { get; private set; } =
					new LDBStructsChildren(noParent); // just easier

		public LDBASTobj() {  //////////////////////////////////////////////////////////////// todo dead?
							  // VStudio thinks this has zero references, which is true explicitly, but it's 
							  // called implicitly on every obj creation - or should do todo- check
			if (recordObjectsCreated) {
				AllCreatedItems.Add(this);
			}
		}


		public virtual void validate() {
			if (reportAlsoIfValidationOK) {
				debugmsg("validating..." + className
					+ " (" + asNameOrIdentContents() + ")");
			}
			mhp_mhk();
			kids.validate();
			// validated = true; -- No, get each subclass to set
			// validated. That forces an explicit call to base() which
			// is more code but ok.
		}


		public bool isValidated { get => validated; }  // todo just => validated;


		public virtual bool isTopLevelItemSeq => false;


		public virtual void mhp_mhk() {  // don't think this need be virtual - todo?
										 // = must have parent, must have kids. The kids may be the
										 // empty set and the parent may be noParent but they must
										 // exist.
										 // Why not just inline this into validate()?
										 // Because a few classes need bespoke validation, but still must check this.
			hardMustParentIsPresent();
			kids.eachKidParentMustBe(this);
		}


		public string asNameOrIdentContents() {
			// for very crude debugging 
			var optName = this is IisNamedItem ni ? ni.name.toRawStr() : "";
			optName = (optName == "" && this is PMQIdent) ? this.toRawStr() : "";

			var optName2 = optName + "  (uid:" + uid.ToString() + ") "
				+ getFirstFewCharsOf(toRawStr());
			return optName2;
		}


		public virtual void getAllUIDs(UIDset uids) {
			uids.Add(this);
			kids.ForEach(namedKid => namedKid.kid.getAllUIDs(uids));
		}


		public bool isStubParent { get => (this is NoParent) || (this is FakeParent); }


		public void validateSpecificItems(params LDBASTobj[] items) {
			Array.ForEach(items, item => item.validate());
		}


		public T? optClimbTo<T>() where T : LDBASTobj {
			var resItem = this;
			while (!(resItem is T)) {
				if (resItem.parent.isStubParent) {
					return null;
				}
				resItem = resItem.parent;
			}

			var res = resItem as T;
			return res;
		}


		public T reqClimbTo<T>() where T : LDBASTobj {
			var loc = className + ".reqClimbTo<T>()";
			var res = optClimbTo<T>();
			hardMust2(!(res is null), 
				// Thanks SO <https://stackoverflow.com/questions/2581642/how-do-i-get-the-type-name-of-a-generic-type-argument>
				// Todo - use this elsewhere, poss ExpectException? todo
				() =>
				loc 
				+ nl + ", climbed to top, didn't find type "
				+ typeof(T).FullName
				+ nl + "Hierarchy is: "
				+ nl + getItemWithParents(this));
			if (res is null) { throw new NNQC(); }
			return res;
		}


		public void hardMustParentIsPresent() {

			hardMust2(!(parent is null), 
				() =>
				 "NULL parent for item of type\n"
				 + $"{GetType()}\n.");

			hardMust2(!(this.parent is NoParent),
				() =>
				"Parent is NoParent on object of type\n"
				+ $"'{this.className}'");
		}


		public void hardMustParentIsAbsent() {
			// For checking cloning - parent should be NoParent immediately 
			// after, before it's set by parent, and never null
			hardMust(!(parent is null),
				 "NULL parent for item of type\n"
				 + $"{GetType()}\n.");
			hardMust((this.parent is NoParent),
				"Parent should be NoParent on object of type\n"
				+ $"'{this.className}'");
		}


		public bool hasParent<P>() {
			hardMust(!(this is NoParent), "Item is NoParent");
			return this.parent is P;
		}


		public bool hasAncestor<A>() {
			hardMust(!this.isStubParent, "Item is NoParent");

			// how to do with a switch expr?
			if (this.parent.isStubParent /* is NoParent || this.parent is FakeParent*/ ) { // todo do as x.hasFudgedParent() - todo
				return false;
			} else if (this.parent is A) {
				return true;
			} else {
				return this.parent.hasAncestor<A>();
			}
		}

		public bool notHasAncestor<A>() =>
			!hasAncestor<A>();


		public void setMyParentTo(LDBASTobj prnt) {
			// An object may only set its parent if its current parent is
			// the dummy 'noParent', which almost every object's parent is
			// set to on creation (see LDBASTobj parent property def).

			// The test for null is peculiar given this. Because
			// noParent is created statically, setting the parent of
			// noParent is difficult, perhaps impossible, to do statically
			// and reliably.Despite the declaration '#nullable enable', it
			// gets created with a null parrent (quite reasonably), hence
			// this check.The actual setting of the parent has to be done
			// at runtime i.e.after the static creation of 'noParent'.

			hardMust2((this.parent is NoParent) || (this.parent is null),
				() =>
				"tried to re-set parent on object of type: "
				+ nl + this.classNameQ
				+ nl + "Original parent type is: "
				+ (this?.parent?.classNameQ == null ? "null" : this.parent.classNameQ)
				+ nl + "and new parent type is: "
				+ prnt.classNameQ
				+ nl + "Item having parent set on it is: "
				// because object may not be fully constructed yet, check howver TODO this is a major hack and must be fixed sometime.
//				+ nl + (this is null ? "(null)" : this.toRawStr()));
				+ nl + this.toRawStr() );

			parent = prnt;
		}


		public void forceNewParentForRootObject() {
			// Every object must have some parent, including the
			// root object of the parse, which is LDBitems
			// (a sequence of LDB items), so this forces one.
			// It is only for that purpose!
			// 
			// LDBitems obj exists for sub objs such as the statements
			// in a while loop, in a proc, in an if... etc., for
			// which their parent is the while/proc/if etc.
			hardMustParentIsAbsent();
			this.parent = fakeParent;
		}


		//public void resetMyParentTo(LDBASTobj prnt) {  ////////////////////////////////// destroy!!!!!! todo
		//	parent = prnt;
		//}

		//////////////////////// xxxxxxxxxxxxx delete - todo
		//private void setParentOnKid(LDBASTobj kid) => // todo needed?
		//											  // check parente doesn't exist - todo
		//	kid.parent = this;

		public virtual void setParentOnKidsAndMakeKidsDict(  // todo - and add to kids dict
															 // this mixes up setting parent and making child dict. Not good. - todo
						params (string kidName, LDBASTobj kid)[] kidsIn) {
			var loc = classNamePlus("setParentOnKidsAndMakeKidsDict");

			var kidsToAdd = new LDBStructsChildren(this, kidsIn);
			hardMust2(kids.isEmpty,
				() =>
				$"setting kids on parent '{className}'" 
				+ "when kids is not empty, "
				+ $"kids is ({kids.Count} kids):"
				+ nl + kids.toDebugStr()
				+ nl + $"Trying to add\n{kidsToAdd.toDebugStr()}\n"
				+ $"in {className}");
			// if there is an LDBStructsChildren and another child,
			// this will blow up - will need to do kids.Add or
			// something - in fact, kids needs to be created in one
			// go, which is what happens.
			kids = kidsToAdd;    ///////////////////////////////////////////////////////// why this? todo
			Array.ForEach(kidsIn, xkid => hardMust(!(xkid.kid is null), loc, "null kid"));
			Array.ForEach(kidsIn, xkid => xkid.kid.setMyParentTo(this));
		}


		public virtual void setParentOnKidsAndAddToKidsDict(
					params (string kidName, LDBASTobj kid)[] kidsIn) {
			var kidsToAdd_justForErrReporting = new LDBStructsChildren(this, kidsIn);
			hardMust2(!kids.isEmpty, // may disable this later 
				() =>
				"adding to kids when kids is empty, "
				 + $"kids is\n{kids.toDebugStr()}\n"
				 + $"Trying to add\n{kidsToAdd_justForErrReporting.toDebugStr()}\n"
				 + $"in {className}");
			Array.ForEach(kidsIn, xkid => {
//				setParentOnKid(xkid.kid);
				xkid.kid.setMyParentTo(this);
				kids.Add(xkid);
			});
		}


		public virtual IdentsUsed getAllIDs() {
		/* This is overridden only in TemplatedAgg, and that's only to 
			deliberately fail. There's something wrong there, todo - 
			un-virtualise this and remove the one override. - todo
		*/
			var iu = new IdentsUsed();
			this._getAllIDs(iu);
			return iu;
		}

		public virtual void _getAllIDs(IdentsUsed iu) {
		// Gets all the identifiers explicitly used (read or written)  ----- todo update this
		// anywhere in the AST.  Does not get idents which are
		// declarations ie. parameters, or var/const declarations
		// unless these are combined with an assignment.
		//
		// Implied idents such as those from tbl.* are implicit and ignored.
		//
		// Root caller supplies new, empty iu
		// need usage here - todo docs//
			kids._getAllIDs(iu);
		}

		public abstract bool canGetAllIdentsUsed {
		// 	This seems to be true on most classes so I could set true 
		// as a default here and override it where it's not wanted, saving 
		// a chunk of code, but I'd rather make it explicit to force me to consider each case. A bit more C&P for less chance of bugs.
			get;
		}


		// Produces a string that should be identical to (excepting
		// whitespaces) the input ie. emit LDB code not valid SQL
		//
		// ToString was causing too many weird problems (due to my
		// inexperience with c#) so just used toRawStr()
		public abstract string toRawStr();

		public override string toDebugStr() => toRawStr();

	} // end LDBASTobj



	public interface IhandlePrefixes<T>
						where T : LDBRoot {

		public bool isPrefixed { get; }

		public bool isExcessPrefixed { get; }

		// The T in addPrefix/rePrefix/withoutPrefix is because we may be 
		// returning eg. a list of items with prefixes 
		// thus modified, not a single PMQIdent

		public T addPrefix(PMQIdent pfx);

		public T rePrefix(PMQIdent newpfx);

//		public T stripPrefix(PMQIdent id);
	}
	// todo - move this





	public abstract class LDBRoot {
		// Root of everything LDB-ish

		public bool Equals([AllowNull] LDBRoot other) {
			throw new LDBShouldNotBeImplementedException(
									"called Equals on LDBRoot");
		}

		public override
			bool Equals([AllowNull] Object other) {
			throw new LDBShouldNotBeImplementedException(
									"called Equals in Object");
		}

		public override int GetHashCode() {
			throw new LDBShouldNotBeImplementedException(
									"called GetHashCode in LDBRoot");
		}

		public static bool operator ==(LDBRoot lhs,
									   LDBRoot rhs) {
			throw new LDBShouldNotBeImplementedException(
									"called == in LDBRoot");
		}

		public static bool operator !=(LDBRoot lhs,
									   LDBRoot rhs) {
			throw new LDBShouldNotBeImplementedException(
									"called != in LDBRoot");
		}


		public virtual string className {
			// todo - this needs checking it works as I want
			// also rename to ldbclassname -- todo
			get => this.GetType().FullName ?? "(unknown)";
		}


		public virtual string classNameQ =>
			"'" + className + "'";


		public virtual string classNamePlus(string plus) =>
			className + "." + plus;

		public override string ToString() {
			// This implicit to-string conversion crap cost me a few hours, so at least
			// do this & blow up at runtime. Would prefer compile time but it seems
			// not possible.  This turns out to have been a good idea,
			// saved me quite a bit more debugging!
			// <https://stackoverflow.com/questions/15345517/is-it-possible-to-disable-implicit-tostring-call>
			// Will try <https://stackoverflow.com/questions/54354342/how-to-get-an-error-shown-when-using-non-strings-as-strings-instead-of-an-autom/54355115#54355115)
			var msg = "LDBRoot.ToString()";
			throw new LDBToStringImplictConversionException(msg);
		}

		public abstract string toDebugStr();
	}



	public abstract class Ephemeral : LDBRoot {
		// for objects that are not part of the AST
	}



	public interface IWithLineNUm {  // todo - use or remove - todo
		//	public 
	}


	public interface IHasTableAlias {
		// Has a table alias AST obj. May be a ...Missing subtype in which
		// it has no alias given by the user, but alias AST is there. More
		// accurately a TableSource alias but ok
		public abstract TableAlias ta { get; set; }
	}


	// Base class for all LDB AST objects
	public abstract class LDBASTobj : LDBRoot {
		// This one is a problem. If I declare the 'parent' property
		// abstract I then have to implement it literally hundreds of
		// times in the derived classes, and I really don't think it
		// would help so just set it to a default which gets overwritten later.
		// 
		// The problem is that those objects were created by the
		// parser, which creates them bottom-up, which means you can't
		// know the parent when they are created. It should be
		// possible to work around this but it does mean reworking the
		// parser fundamentally, and I'm not going to do that
		// assumeing it's even possible.

		// todo - read up on priv/pub/prog accessors I really don't
		// get these at all
		public LDBASTobj parent { get; private set; }
						= noParent;
		// Unique ID per object, useful for distinguishing them
		// without general equality.
		readonly public ulong uid = newUID();

		// public abstract int lineNum { get; set; }

		// Is validated is a cheapish check to ensure everything gets
		// validated
		// can do this as auto prop? todo
		private bool _validated = false;

		protected bool validated {
			get => _validated;
			set {
				// hardMust(!validated, "... already validated... "); - no
				// It's hard to avoid validating a thing twice, so for now allow it
				// Look into it later, really shouldn't happen
				_validated = true;
			}
		}

		public LDBStructsChildren kids { get; private set; } =
					new LDBStructsChildren(noParent); // just easier

		public LDBASTobj() {  //////////////////////////////////////////////////////////////// todo dead?
							  // VStudio thinks this has zero references, which is true explicitly, but it's 
							  // called implicitly on every obj creation - or should do todo- check
			if (recordObjectsCreated) {
				AllCreatedItems.Add(this);
			}
		}


		public virtual void validate() {
			if (reportAlsoIfValidationOK) {
				debugmsg("validating..." + className
					+ " (" + asNameOrIdentContents() + ")");
			}
			mhp_mhk();
			kids.validate();
			// validated = true; -- No, get each subclass to set
			// validated. That forces an explicit call to base() which
			// is more code but ok.
		}


		public bool isValidated { get => validated; }  // todo just => validated;


		public virtual bool isTopLevelItemSeq => false;


		public virtual void mhp_mhk() {  // don't think this need be virtual - todo?
										 // = must have parent, must have kids. The kids may be the
										 // empty set and the parent may be noParent but they must
										 // exist.
										 // Why not just inline this into validate()?
										 // Because a few classes need bespoke validation, but still must check this.
			hardMustParentIsPresent();
			kids.eachKidParentMustBe(this);
		}


		public string asNameOrIdentContents() {
			// for very crude debugging 
			var optName = this is IisNamedItem ni ? ni.name.toRawStr() : "";
			optName = (optName == "" && this is PMQIdent) ? this.toRawStr() : "";

			var optName2 = optName + "  (uid:" + uid.ToString() + ") "
				+ getFirstFewCharsOf(toRawStr());
			return optName2;
		}


		public virtual void getAllUIDs(UIDset uids) {
			uids.Add(this);
			kids.ForEach(namedKid => namedKid.kid.getAllUIDs(uids));
		}


		public bool isStubParent { get => (this is NoParent) || (this is FakeParent); }


		public void validateSpecificItems(params LDBASTobj[] items) {
			Array.ForEach(items, item => item.validate());
		}


		public T? optClimbTo<T>() where T : LDBASTobj {
			var resItem = this;
			while (!(resItem is T)) {
				if (resItem.parent.isStubParent) {
					return null;
				}
				resItem = resItem.parent;
			}

			var res = resItem as T;
			return res;
		}


		public T reqClimbTo<T>() where T : LDBASTobj {
			var loc = className + ".reqClimbTo<T>()";
			var res = optClimbTo<T>();
			hardMust2(!(res is null), 
				// Thanks SO <https://stackoverflow.com/questions/2581642/how-do-i-get-the-type-name-of-a-generic-type-argument>
				// Todo - use this elsewhere, poss ExpectException? todo
				() =>
				loc 
				+ nl + ", climbed to top, didn't find type "
				+ typeof(T).FullName
				+ nl + "Hierarchy is: "
				+ nl + getItemWithParents(this));
			if (res is null) { throw new NNQC(); }
			return res;
		}


		public void hardMustParentIsPresent() {

			hardMust2(!(parent is null), 
				() =>
				 "NULL parent for item of type\n"
				 + $"{GetType()}\n.");

			hardMust2(!(this.parent is NoParent),
				() =>
				"Parent is NoParent on object of type\n"
				+ $"'{this.className}'");
		}


		public void hardMustParentIsAbsent() {
			// For checking cloning - parent should be NoParent immediately 
			// after, before it's set by parent, and never null
			hardMust(!(parent is null),
				 "NULL parent for item of type\n"
				 + $"{GetType()}\n.");
			hardMust((this.parent is NoParent),
				"Parent should be NoParent on object of type\n"
				+ $"'{this.className}'");
		}


		public bool hasParent<P>() {
			hardMust(!(this is NoParent), "Item is NoParent");
			return this.parent is P;
		}


		public bool hasAncestor<A>() {
			hardMust(!this.isStubParent, "Item is NoParent");

			// how to do with a switch expr?
			if (this.parent.isStubParent /* is NoParent || this.parent is FakeParent*/ ) { // todo do as x.hasFudgedParent() - todo
				return false;
			} else if (this.parent is A) {
				return true;
			} else {
				return this.parent.hasAncestor<A>();
			}
		}

		public bool notHasAncestor<A>() =>
			!hasAncestor<A>();


		public void setMyParentTo(LDBASTobj prnt) {
			// An object may only set its parent if its current parent is
			// the dummy 'noParent', which almost every object's parent is
			// set to on creation (see LDBASTobj parent property def).

			// The test for null is peculiar given this. Because
			// noParent is created statically, setting the parent of
			// noParent is difficult, perhaps impossible, to do statically
			// and reliably.Despite the declaration '#nullable enable', it
			// gets created with a null parrent (quite reasonably), hence
			// this check.The actual setting of the parent has to be done
			// at runtime i.e.after the static creation of 'noParent'.

			hardMust2((this.parent is NoParent) || (this.parent is null),
				() =>
				"tried to re-set parent on object of type: "
				+ nl + this.classNameQ
				+ nl + "Original parent type is: "
				+ (this?.parent?.classNameQ == null ? "null" : this.parent.classNameQ)
				+ nl + "and new parent type is: "
				+ prnt.classNameQ
				+ nl + "Item having parent set on it is: "
				// because object may not be fully constructed yet, check howver TODO this is a major hack and must be fixed sometime.
//				+ nl + (this is null ? "(null)" : this.toRawStr()));
				+ nl + this.toRawStr() );

			parent = prnt;
		}


		public void forceNewParentForRootObject() {
			// Every object must have some parent, including the
			// root object of the parse, which is LDBitems
			// (a sequence of LDB items), so this forces one.
			// It is only for that purpose!
			// 
			// LDBitems obj exists for sub objs such as the statements
			// in a while loop, in a proc, in an if... etc., for
			// which their parent is the while/proc/if etc.
			hardMustParentIsAbsent();
			this.parent = fakeParent;
		}


		//public void resetMyParentTo(LDBASTobj prnt) {  ////////////////////////////////// destroy!!!!!! todo
		//	parent = prnt;
		//}

		//////////////////////// xxxxxxxxxxxxx delete - todo
		//private void setParentOnKid(LDBASTobj kid) => // todo needed?
		//											  // check parente doesn't exist - todo
		//	kid.parent = this;

		public virtual void setParentOnKidsAndMakeKidsDict(  // todo - and add to kids dict
															 // this mixes up setting parent and making child dict. Not good. - todo
						params (string kidName, LDBASTobj kid)[] kidsIn) {
			var loc = classNamePlus("setParentOnKidsAndMakeKidsDict");

			var kidsToAdd = new LDBStructsChildren(this, kidsIn);
			hardMust2(kids.isEmpty,
				() =>
				$"setting kids on parent '{className}'" 
				+ "when kids is not empty, "
				+ $"kids is ({kids.Count} kids):"
				+ nl + kids.toDebugStr()
				+ nl + $"Trying to add\n{kidsToAdd.toDebugStr()}\n"
				+ $"in {className}");
			// if there is an LDBStructsChildren and another child,
			// this will blow up - will need to do kids.Add or
			// something - in fact, kids needs to be created in one
			// go, which is what happens.
			kids = kidsToAdd;    ///////////////////////////////////////////////////////// why this? todo
			Array.ForEach(kidsIn, xkid => hardMust(!(xkid.kid is null), loc, "null kid"));
			Array.ForEach(kidsIn, xkid => xkid.kid.setMyParentTo(this));
		}


		public virtual void setParentOnKidsAndAddToKidsDict(
					params (string kidName, LDBASTobj kid)[] kidsIn) {
			var kidsToAdd_justForErrReporting = new LDBStructsChildren(this, kidsIn);
			hardMust2(!kids.isEmpty, // may disable this later 
				() =>
				"adding to kids when kids is empty, "
				 + $"kids is\n{kids.toDebugStr()}\n"
				 + $"Trying to add\n{kidsToAdd_justForErrReporting.toDebugStr()}\n"
				 + $"in {className}");
			Array.ForEach(kidsIn, xkid => {
//				setParentOnKid(xkid.kid);
				xkid.kid.setMyParentTo(this);
				kids.Add(xkid);
			});
		}


		public virtual IdentsUsed getAllIDs() {
		/* This is overridden only in TemplatedAgg, and that's only to 
			deliberately fail. There's something wrong there, todo - 
			un-virtualise this and remove the one override. - todo
		*/
			var iu = new IdentsUsed();
			this._getAllIDs(iu);
			return iu;
		}

		public virtual void _getAllIDs(IdentsUsed iu) {
		// Gets all the identifiers explicitly used (read or written)  ----- todo update this
		// anywhere in the AST.  Does not get idents which are
		// declarations ie. parameters, or var/const declarations
		// unless these are combined with an assignment.
		//
		// Implied idents such as those from tbl.* are implicit and ignored.
		//
		// Root caller supplies new, empty iu
		// need usage here - todo docs//
			kids._getAllIDs(iu);
		}

		public abstract bool canGetAllIdentsUsed {
		// 	This seems to be true on most classes so I could set true 
		// as a default here and override it where it's not wanted, saving 
		// a chunk of code, but I'd rather make it explicit to force me to consider each case. A bit more C&P for less chance of bugs.
			get;
		}


		// Produces a string that should be identical to (excepting
		// whitespaces) the input ie. emit LDB code not valid SQL
		//
		// ToString was causing too many weird problems (due to my
		// inexperience with c#) so just used toRawStr()
		public abstract string toRawStr();

		public override string toDebugStr() => toRawStr();

	} // end LDBASTobj


	
}

bug#48871: 27.2; Unusably slow in C# mode

[Eli Zaretskii]

> From: jan <rtm443x@googlemail.com>
> Date: Sun, 6 Jun 2021 15:53:11 +0100
> Cc: 48871@debbugs.gnu.org
> 
> I don't recall installing c# but may well have happened.
> >From package-list-packages:
> 
> csharp-mode        20210328.2004 installed             C# mode derived mode
> 
> Which does not say built-in so likely I did. Looking in the unzipped
> emacs 27.2 , no relevant *sharp* file in it, and did find it in the
> elpa directory, so I guess must have.
> 
> Started with -Q.  Did the M-x load-file for csharp-mode.el (FYI also
> had to do load-file for csharp-compilation.el before that to make it
> happy).
> 
> Finally got to load the C# file itself, exactly the same. No faster.
> 
> Troublesome C# file attached.

Thanks.

Alan, can you look into this?  It could be some problem in
csharp-mode, but the profiler says 97% of the time is spent in a CC
mode code, so maybe you can shed some light on this?

I see that almost the entire 6896-line file is enclosed in a single
"namespace LDB { ... }" block, maybe this is the reason?

Here's the main portion of a profile measured on my system from just
inserting 3 characters at BOB of the file attached by the OP.  An
unoptimized build of Emacs 28 took about 2 min(!) to process those 3
self-inserting characters.

	  8693  98% - command-execute
	  8693  98%  - call-interactively
	  8693  98%   - funcall-interactively
	  8690  98%    - self-insert-command
	  8627  98%     - c-before-change
	  8627  98%      - mapc
	  8627  98%       - #<compiled 0x1aaa1ff2c62de223>
	  8627  98%        - c-before-change-check-unbalanced-strings
	  8611  97%           c-pps-to-string-delim
	    10   0%         - c-syntactic-re-search-forward
	     2   0%          - c-beginning-of-macro
	     2   0%             back-to-indentation
	     1   0%            #<compiled 0x4f47ef635173>
	     1   0%           c-clear-syn-tab
	    63   0%     - c-after-change

bug#48871: 27.2; Unusably slow in C# mode

[jan via Bug reports for GNU Emacs, the Swiss army knife of text editors]

Looks like strings doing it.

Go to fundamental mode (or search&replace will take forever),
completely remove all strings with regex replace:

".*?" ->

Save file then close and reopen emacs, reopen file, ensure it's in
csharp mode, char insertion is almost back to normal.

That is, remove strings entirely. Just replacing them with empty
strings "" didn't help.

cheers

jan




On 06/06/2021, Eli Zaretskii <eliz@gnu.org> wrote:
>> From: jan <rtm443x@googlemail.com>
>> Date: Sun, 6 Jun 2021 15:53:11 +0100
>> Cc: 48871@debbugs.gnu.org
>>
>> I don't recall installing c# but may well have happened.
>> >From package-list-packages:
>>
>> csharp-mode        20210328.2004 installed             C# mode derived
>> mode
>>
>> Which does not say built-in so likely I did. Looking in the unzipped
>> emacs 27.2 , no relevant *sharp* file in it, and did find it in the
>> elpa directory, so I guess must have.
>>
>> Started with -Q.  Did the M-x load-file for csharp-mode.el (FYI also
>> had to do load-file for csharp-compilation.el before that to make it
>> happy).
>>
>> Finally got to load the C# file itself, exactly the same. No faster.
>>
>> Troublesome C# file attached.
>
> Thanks.
>
> Alan, can you look into this?  It could be some problem in
> csharp-mode, but the profiler says 97% of the time is spent in a CC
> mode code, so maybe you can shed some light on this?
>
> I see that almost the entire 6896-line file is enclosed in a single
> "namespace LDB { ... }" block, maybe this is the reason?
>
> Here's the main portion of a profile measured on my system from just
> inserting 3 characters at BOB of the file attached by the OP.  An
> unoptimized build of Emacs 28 took about 2 min(!) to process those 3
> self-inserting characters.
>
> 	  8693  98% - command-execute
> 	  8693  98%  - call-interactively
> 	  8693  98%   - funcall-interactively
> 	  8690  98%    - self-insert-command
> 	  8627  98%     - c-before-change
> 	  8627  98%      - mapc
> 	  8627  98%       - #<compiled 0x1aaa1ff2c62de223>
> 	  8627  98%        - c-before-change-check-unbalanced-strings
> 	  8611  97%           c-pps-to-string-delim
> 	    10   0%         - c-syntactic-re-search-forward
> 	     2   0%          - c-beginning-of-macro
> 	     2   0%             back-to-indentation
> 	     1   0%            #<compiled 0x4f47ef635173>
> 	     1   0%           c-clear-syn-tab
> 	    63   0%     - c-after-change
>

bug#48871: 27.2; Unusably slow in C# mode

[Eli Zaretskii]

> From: jan <rtm443x@googlemail.com>
> Date: Sun, 6 Jun 2021 19:00:57 +0100
> Cc: Alan Mackenzie <acm@muc.de>, 48871@debbugs.gnu.org
> 
> Looks like strings doing it.

Well, the fact that c-pps-to-string-delim is the hot spot kinda says
that...

bug#48871: 27.2; Unusably slow in C# mode

[jan via Bug reports for GNU Emacs, the Swiss army knife of text editors]

Ah... well... yeah...

searching for "c-pps-to-string-delim" finds this
<https://github.com/emacs-csharp/csharp-mode/issues/207>, similar to
what I referenced in my initial post.
Neither are very optimistic. If this is a wontfix I'll understand.

cheers

jan


On 06/06/2021, Eli Zaretskii <eliz@gnu.org> wrote:
>> From: jan <rtm443x@googlemail.com>
>> Date: Sun, 6 Jun 2021 19:00:57 +0100
>> Cc: Alan Mackenzie <acm@muc.de>, 48871@debbugs.gnu.org
>>
>> Looks like strings doing it.
>
> Well, the fact that c-pps-to-string-delim is the hot spot kinda says
> that...
>

bug#48871: 27.2; Unusably slow in C# mode

[Alan Mackenzie]

Hello, Jan and Eli.

On Sun, Jun 06, 2021 at 19:22:17 +0100, jan wrote:
> Ah... well... yeah...

> searching for "c-pps-to-string-delim" finds this
> <https://github.com/emacs-csharp/csharp-mode/issues/207>, similar to
> what I referenced in my initial post.
> Neither are very optimistic. If this is a wontfix I'll understand.

This is indeed a string problem in CC Mode.  More precisely, it's a
multi-line string problem, such being started in C# mode by @".  For
some very bad reason (which I've got written down somewhere) CC Mode is
scanning to the end of the buffer for each character inserted.  Sorry.

As a workaround, if you don't have any multi-line strings in your
source, put the following in your csharp-mode-hook:

    (setq c-multiline-string-start-char nil)

..  That should get rid of the excessive scanning for the time being.

I've spent much of the last few weeks adapting the C++ raw string
mechanism also to handle multi-line strings in other languages such as
C#.  I'm hoping to have it up and running soon.  Then, perhaps, the C#
Mode maintainer will incorporate it into C# Mode, which should be a
straightforward task.

> cheers

> jan


> On 06/06/2021, Eli Zaretskii <eliz@gnu.org> wrote:
> >> From: jan <rtm443x@googlemail.com>
> >> Date: Sun, 6 Jun 2021 19:00:57 +0100
> >> Cc: Alan Mackenzie <acm@muc.de>, 48871@debbugs.gnu.org

> >> Looks like strings doing it.

> > Well, the fact that c-pps-to-string-delim is the hot spot kinda says
> > that...

-- 
Alan Mackenzie (Nuremberg, Germany).

bug#48871: 27.2; Unusably slow in C# mode

[jan via Bug reports for GNU Emacs, the Swiss army knife of text editors]

I forgot to provide feedback to this list, this workaround restored performance.

jan

(Thanks to Alan Mackenzie for this, and his continuing work to fix the
underlying problem)

On 06/06/2021, Alan Mackenzie <acm@muc.de> wrote:
> Hello, Jan and Eli.
>
> On Sun, Jun 06, 2021 at 19:22:17 +0100, jan wrote:
>> Ah... well... yeah...
>
>> searching for "c-pps-to-string-delim" finds this
>> <https://github.com/emacs-csharp/csharp-mode/issues/207>, similar to
>> what I referenced in my initial post.
>> Neither are very optimistic. If this is a wontfix I'll understand.
>
> This is indeed a string problem in CC Mode.  More precisely, it's a
> multi-line string problem, such being started in C# mode by @".  For
> some very bad reason (which I've got written down somewhere) CC Mode is
> scanning to the end of the buffer for each character inserted.  Sorry.
>
> As a workaround, if you don't have any multi-line strings in your
> source, put the following in your csharp-mode-hook:
>
>     (setq c-multiline-string-start-char nil)
>
> ..  That should get rid of the excessive scanning for the time being.
>
> I've spent much of the last few weeks adapting the C++ raw string
> mechanism also to handle multi-line strings in other languages such as
> C#.  I'm hoping to have it up and running soon.  Then, perhaps, the C#
> Mode maintainer will incorporate it into C# Mode, which should be a
> straightforward task.
>
>> cheers
>
>> jan
>
>
>> On 06/06/2021, Eli Zaretskii <eliz@gnu.org> wrote:
>> >> From: jan <rtm443x@googlemail.com>
>> >> Date: Sun, 6 Jun 2021 19:00:57 +0100
>> >> Cc: Alan Mackenzie <acm@muc.de>, 48871@debbugs.gnu.org
>
>> >> Looks like strings doing it.
>
>> > Well, the fact that c-pps-to-string-delim is the hot spot kinda says
>> > that...
>
> --
> Alan Mackenzie (Nuremberg, Germany).
>