Powershell Parse Response String of Requirements.txt file

user_9485834 0
Viewed 153

Been struggling for a few hours. I'm trying to hit this link That has these contents:

# Generated on: Python 3.6.12
# With snowflake-connector-python version: 2.4.6
asn1crypto==1.4.0
azure-common==1.1.27
azure-core==1.15.0
azure-storage-blob==12.8.1
boto3==1.17.98
botocore==1.20.98
certifi==2021.5.30
cffi==1.14.5
chardet==4.0.0
cryptography==3.4.7
dataclasses==0.8
idna==2.10
isodate==0.6.0
jmespath==0.10.0
msrest==0.6.21
oauthlib==3.1.1
oscrypto==1.2.1
pycparser==2.20
pycryptodomex==3.10.1
PyJWT==2.1.0
pyOpenSSL==20.0.1
python-dateutil==2.8.1
pytz==2021.1
requests==2.25.1
requests-oauthlib==1.3.0
s3transfer==0.4.2
six==1.16.0
urllib3==1.26.5

And pass each non-commented out line to Poetry (a command line tool for Python dependency management)

This is my first step

(iwr https://raw.githubusercontent.com/snowflakedb/snowflake-connector-python/v2.5.1/tested_requirements/requirements_36.reqs | Select-Object).Content > req.txt

Where I'm struggling is I've tried doing convertfrom-string with various delimiters, and .Split(), and I can't seem to parse out the pieces I need. Poetry needs to take as input just the "packagename==" although version number is optional. So I essentially want to ignore lines that start with a "#" and then pass each line through a pipe, or even save it as an array. It doesn't seem to respond to setting a delimited to "\t" or carriage return "`r".

So next I would do something like

foreach($package in $package_array){poetry add $package}

Any help would be appreciated.

python-poetry powershell
1 Answers

AdminOfThings provided a good pointer in a comment, but let me try to put it all together:

$url = 'https://raw.githubusercontent.com/snowflakedb/snowflake-connector-python/v2.5.1/tested_requirements/requirements_36.reqs'

foreach ($pkgLine in (irm $url).Trim() -split '\r?\n' -notmatch '^\s*#') {
  # Remove `Write-Host` to perform the actual poetry call.
  Write-Host poetry add ($pkgLine -replace '=.*')
}

  • irm is the built-in alias for Invoke-RestMethod, which is a simpler alternative to Invoke-WebRequest (iwr) in this case, because it directly returns the text of interest, as a multi-line string.

    • As an aside: the | Select-Object in your code is effectively a no-op and can be omitted.

  • .Trim() trims a trailing newline (all trailing whitespace).

  • -split '\r?\n' splits the string into individual lines.

  • -notmatch '^\s*# filters out all lines that start with #, optionally preceded by whitespace.

  • -replace '=.* removes everything starting with = from each package line.

user_453750
Related