Wrap Web.Contents so functions using it aren't excluded from M.pq - #18
Open
wakinniranye31 wants to merge 1 commit into
Open
Wrap Web.Contents so functions using it aren't excluded from M.pq#18wakinniranye31 wants to merge 1 commit into
wakinniranye31 wants to merge 1 commit into
Conversation
Web.Contents was excluded entirely from the generated Expression.Evaluate
environment because passing it in directly (Web.Contents = Web.Contents)
triggers the 'can't use a Dynamic Data Source' Formula.Firewall error.
That meant any custom function using Web.Contents internally (e.g.
GetSPList, GitHubRepoFiles, Countries) would throw during evaluation and
silently fall back to displaying as raw text.
Instead of excluding Web.Contents, map it to a same-signature wrapper
function defined in M.pq's own let block:
#"Web.Contents wrapper" = (url as text, optional options as
nullable record) as any => Web.Contents(url, options)
Passing the wrapper (rather than Web.Contents itself) through
Expression.Evaluate's environment is the standard community workaround
for this exact Formula.Firewall limitation.
Closes OscarValerock#1
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Closes #1.
Root cause
M.pqloads each function file's source and runs it throughExpression.Evaluate(source, environment).Web.Contentsis deliberately left out of thatenvironmentrecord (see the old comment inM_Creator.py'sexclude_strings) because passing it in directly asWeb.Contents = Web.Contentstriggers Power Query's "This expression is dynamic, so it might not work..." / Formula.Firewall "can't use a Dynamic Data Source" error, especially on refresh from the Power BI service — this is a well-known M limitation onceWeb.Contentsis passed around as a first-class value rather than called literally (see the reference already linked inFunctions/SharePoint/GetSPList.pq's own docs: community.powerbi.com/.../Dynamic-data-sources-aren-t-refreshed...).Because
Web.Contentsis excluded, any function that calls it internally —GetSPList,GitHubRepoFiles,Countries, the OpenAI functions — fails insideExpression.Evaluatewith an undefined-identifier error, which thetry ... otherwisefallback inM_Creator.py's template swallows by falling back to returning the function's raw text instead of an evaluated function. That's exactly the "displays as text" symptom in the issue.Fix
Rather than passing
Web.Contentsitself into the environment, map it to a locally-defined wrapper with the same signature:and register
Web.Contents = #"Web.Contents wrapper"in theExpression.Evaluateenvironment instead of the bare function. This is the standard community-documented workaround for this exact Formula.Firewall limitation — wrapping aWeb.Contentscall in a named function whose parameters flow straight through to the call often lets Power Query's static dependency analysis "see" the literal call site again.Changes
M_Creator.py:manual_mappingsdict for identifiers that need a custom expression rather than the auto-generatedName = Name, and wiredWeb.Contentsinto it.#"Web.Contents wrapper"step to theM_Codetemplate.//,Web.Contents = Web.Contents //...comment, replaced with an accurate comment on whyWeb.Contentsstays inexclude_strings.",\n".join(...)so appendingmanual_mappingsentries didn't need a second near-duplicate loop.M.pq: regenerated viapython M_Creator.pyso it matches whatM_Creator.pynow produces.Verification
M_Creator.pyand confirmed the generatedM.pqcontainsWeb.Contents = #"Web.Contents wrapper"in theExpression.Evaluateenvironment, and the wrapper step itself, in the right scope to be visible there.M.pq(all balanced) as a basic sanity check.GetSPListnow evaluates correctly instead of falling back to text — happy to iterate if it needs adjustment.