source precheck.sh errors #16

kaster-san · 2023-08-07T17:18:59Z

when i reached source precheck.sh i got these errors :

: command not found
E: Invalid operation update
: command not found
-bash: precheck.sh: line 14: syntax error near unexpected token `$'do\r''
'bash: precheck.sh: line 14: `for command in "${commands[@]}"; do

here is the content of precheck.sh

#!/usr/bin/env bash

sudo apt -qq update

commands=(patch sed python3-pip)
python_ver=$(whereis python3 | grep -Eo 'python3\.[0-9]+ ' | sort -u | tail -n 1|xargs)
if [ -z "$python_ver" ]; then
    # Get latest version of python 3 available in the repository
    python_ver=$(apt search -qq '^python3\.[0-9]+$' 2>/dev/null| grep -Eo 'python3\.[0-9]+' | grep -Fv python3.11 | tail -n 1)
    commands+=($python_ver)
fi
commands+=($python_ver-venv)

for command in "${commands[@]}"; do
    if [ ! -n "$(command -v $command)" ]; then
        if [ -n "$(command -v apt)" ]; then
            sudo apt install -y -qq $command
        else
            sudo yum install -y $command
        fi
    fi
done

# Note: If a wrong version of pip is installed
# Use below commands to install correct pip version
# if [ ! -n "$(command -v pip3.8)" ]
# then
#     wget -q https://bootstrap.pypa.io/get-pip.py -O /tmp/get-pip.py
#     sudo $python_ver /tmp/get-pip.py
#     rm /tmp/get-pip.py
# fi

cd /cake_fuzzer
sudo pip3 install -q --upgrade virtualenv
sudo virtualenv -q -p $python_ver venv
if [ ! -e venv ]; then
    $python_ver -m venv venv
fi
source venv/bin/activate

pip install -qr requirements.txt

echo "setup finished!"

can anyone point the reason ?

update : here is the fix :

 sudo apt install dos2unix
 dos2unix precheck.sh

The text was updated successfully, but these errors were encountered:

dawid-czarnecki · 2023-08-23T07:54:28Z

@kaster-san This must be some issue due to editors or when you were transferring the file.
All files are prepared in and designed for linux environments so should be no need to convert to unix bytes.

sreegopcs · 2024-04-01T13:45:42Z

python cake_fuzzer.py instrument check
[-] Error detected by app_info: {'type': 'error', 'message': '() '}

how to resolve this error......i had made the setup as mentioned in the readme file

dawid-czarnecki · 2024-04-02T11:17:42Z

Can you describe your setup?

sreegopcs · 2024-04-02T15:45:21Z

Actually i had used ubuntu for the setup as host and used vmware on the top to launch misp ....and used the step specified in the readme.....intially i faced many errors....i resolved it...but now i am stuck with this error

dawid-czarnecki · 2024-04-04T11:37:11Z

If you use python3.6 it might not work because it doesn't have all the modules required (although it would be a different error than the one you see). Try python 3.8 or newer. Check:

if your config points correctly to webroot of MISP (by default: /var/www/MISP/app/webroot/)
if you are inside of the virutal environment when you run the command

I've just setup similar environment using MISP VMWare images, updated it, setup cake_fuzzer with python 3.8 and instrument check works fine:

(venv) root@misp:/cake_fuzzer# python3.8 cake_fuzzer.py instrument check
Applied / Unapplied
FunctionCall Renames: x/18
Patches: 0/6
Copies: 0/0
Annotations: x/1
php.ini updates: x/1

sreegopcs · 2024-04-07T06:25:58Z

Hey, Thank you for your quick reply... I have a concern that, whether it is possible to use Windows as the host, As when I try to use like that, it is showing OS error while creating then venv. Also if it is possible to share a report or final result of the tool, it will be helpful for me. Looking forward to hearing from you and contribute to this futuristic project.

…

On Thu, 4 Apr, 2024, 5:07 pm Dawid, ***@***.***> wrote: If you use python3.6 it might not work because it doesn't have all the modules required (although it would be a different error than the one you see). Try python 3.8 or newer. Check: - if your config points correctly to webroot of MISP (by default: /var/www/MISP/app/webroot/) - if you are inside of the virutal environment when you run the command I've just setup similar environment using MISP VMWare images, updated it, setup cake_fuzzer with python 3.8 and instrument check works fine: (venv) ***@***.***:/cake_fuzzer# python3.8 cake_fuzzer.py instrument check Applied / Unapplied FunctionCall Renames: x/18 Patches: 0/6 Copies: 0/0 Annotations: x/1 php.ini updates: x/1 — Reply to this email directly, view it on GitHub <#16 (comment)>, or unsubscribe <https://github.com/notifications/unsubscribe-auth/AZL5KOKC2QF4SZTMBFLQO6LY3U3PXAVCNFSM6AAAAAA3HJIBEKVHI2DSMVQWIX3LMV43OSLTON2WKQ3PNVWWK3TUHMZDAMZWHEZTCMJTHE> . You are receiving this because you commented.Message ID: ***@***.***>

dawid-czarnecki · 2024-04-12T13:31:31Z

@sreegopcs Yes, it's possible to use windows as the host. If you setup CakeFuzzer on guest linux, it doesn't touch the host at all (unless you setup development environment with file sharing between host&guest).

Regarding the output. Here is the example output of different stages of Cake Fuzzer:

Here is the instrumentation stripped (It's a long process with a lot of output):

(venv) root@misp:/cake_fuzzer# python3.8 cake_fuzzer.py instrument apply
Patches Applied 6                                             
Copies Applied 0                     
Function calls to 'header' renamed to '__cakefuzzer_header' in '/var/www/MISP/app/../app/Lib/cakephp/lib/Cake/Network/CakeResponse.php', original file renamed to '/var/www/MISP/app/../app/Lib/cakephp/lib/Cake/Network/CakeResponse.php.prerename'                                                                                                                  
Function calls to 'header' renamed to '__cakefuzzer_header' in '/var/www/MISP/app/../app/Lib/cakephp/lib/Cake/TestSuite/Reporter/CakeTextReporter.php', original file renamed to '/var/www/MISP/app/../app/Lib/cakephp/lib/Cake/TestSuite/Repo
rter/CakeTextReporter.php.prerename'
Function calls to 'header' renamed to '__cakefuzzer_header' in '/var/www/MISP/app/../app/Lib/cakephp/lib/Cake/TestSuite/Reporter/CakeHtmlReporter.php', original file renamed to '/var/www/MISP/app/../app/Lib/cakephp/lib/Cake/TestSuite/Repo
rter/CakeHtmlReporter.php.prerename'
...
Type hints and annotations removed from '/var/www/MISP/app/../app/View/Helper/AclHelper.php', original file renamed to '/var/www/MISP/app/../app/View/Helper/AclHelper.php.preannotation'
Annotations Removed 1
php.ini updates applied 1

Here is the scheduling process:

(venv) root@misp:/cake_fuzzer# python3.8 cake_fuzzer.py run fuzzer
created all that's necessary
discovered 2 files to scan with total of 627 paths
Scheduled SSRFAttackStrategy: 627 attacks, 1 scanners.
Scheduled LFIAttackStrategy: 1254 attacks, 2 scanners.
Scheduled DeserializeAttackStrategy: 1881 attacks, 4 scanners.
Scheduled RFIAttackStrategy: 627 attacks, 6 scanners.
Scheduled SQLInjectionAttackStrategy: 1254 attacks, 3 scanners.
Scheduled XSSAttackStrategy: 4389 attacks, 5 scanners.
Scheduled CommandInjectionAttackStrategy: 7524 attacks, 5 scanners.
Scheduled PhpCodeInjectionAttackStrategy: 627 attacks, 4 scanners.
Scheduled SSTIAttackStrategy: 627 attacks, 1 scanners.
DONE!
Finished!

Here are the periodic monitors working (constantly until you kill the process):

(venv) root@misp:/cake_fuzzer# python3.8 cake_fuzzer.py run periodic_monitors
created all that's necessary
created all that's necessary
Scanning each 0.5s \

Here are the iteration monitors working (constantly until you kill the process):

(venv) root@misp:/cake_fuzzer# python3.8 cake_fuzzer.py run iteration_monitors
created all that's necessary
created all that's necessary
Scanning Iteration Results |                                | 0/0 - 0.0% - eta: 00:00:00 - elapsed: 9

Here is the final scanning/attacking process (once it's finished you need to kill it as well):

(venv) root@misp:/cake_fuzzer# python3.8 cake_fuzzer.py run attack_queue
created all that's necessary
Executing Attack Scenarios |                                | 3/18807 - 0.0% - eta: 18:50:29 - elapsed: 10

The registry creates the results.json file with detected vulnerabilities:

(venv) root@misp:/cake_fuzzer# python3.8 cake_fuzzer.py run registry; jq length results.json
created all that's necessary
created all that's necessary
created all that's necessary
Finished!
0

sreegopcs · 2024-04-21T17:33:21Z

(venv) root@misp:/cake_fuzzer# python3.8 cake_fuzzer.py instrument check
Applied / Unapplied
FunctionCall Renames: x/18
Patches: 6/2
Copies: 0/1
Annotations: x/1
php.ini updates: x/1
(venv) root@misp:/cake_fuzzer# python3.8 cake_fuzzer.py instrument apply
Traceback (most recent call last):

File "cake_fuzzer.py", line 574, in
app()

File "cake_fuzzer.py", line 564, in instrumentation
asyncio.run(apply_instrumentation())

File "/usr/lib/python3.8/asyncio/runners.py", line 43, in run
return loop.run_until_complete(main)

File "/usr/lib/python3.8/asyncio/base_events.py", line 608, in run_until_complete
return future.result()

File "cake_fuzzer.py", line 510, in apply_instrumentation
await inst.apply()

File "/cake_fuzzer/cakefuzzer/instrumentation/instrumentator.py", line 165, in apply
unapplied = await apply(*unapplied)

File "/cake_fuzzer/cakefuzzer/instrumentation/init.py", line 32, in apply
await asyncio.gather(*[p.apply(semaphore) for p in args])

File "/cake_fuzzer/cakefuzzer/instrumentation/patch.py", line 45, in apply
await _run_subprocess("patch", str(self.original), str(self.patch))

File "/cake_fuzzer/cakefuzzer/instrumentation/patch.py", line 19, in _run_subprocess
raise InstrumentationError(

cakefuzzer.instrumentation.InstrumentationError: Error: Error while instrumenting, got non-zero response from subprocess
Try: patch /var/www/MISP/app/../vendor/cakephp/authentication/src/AuthenticationService.php cakefuzzer/instrumentation/patches/CakePHP/4/vendor/cakephp/authentication/src/AuthenticationService.php.patch

(venv) root@misp:/cake_fuzzer#

any sugestions

dawid-czarnecki · 2024-04-23T17:02:36Z

What version of MISP are you trying to scan?

sreegopcs · 2024-04-26T06:43:56Z

MISP_v2.4.146@0c25b72 is version i am using

dawid-czarnecki · 2024-04-28T16:26:31Z

You need to set the instrumentation to CAKE version 2 (MISP is built on CAKE 2).
You can do this in config/config.ini. Set the following env var:

INSTRUMENTATION_INI="config/instrumentation_cake2.ini"

sreegopcs · 2024-05-01T18:42:47Z

(venv) root@misp:/cake_fuzzer# python3.8 cake_fuzzer.py instrument apply
Patches Applied 0
Copies Applied 0
PHP Fatal error: Uncaught Error: Call to undefined method PhpParser\ParserFactory::create() in /cake_fuzzer/cakefuzzer/phpfiles/instrumentation/rename_function_call.php:62
Stack trace:
#0 {main}
thrown in /cake_fuzzer/cakefuzzer/phpfiles/instrumentation/rename_function_call.php on line 62
Traceback (most recent call last):

File "cake_fuzzer.py", line 574, in
app()

File "cake_fuzzer.py", line 564, in instrumentation
asyncio.run(apply_instrumentation())

File "/usr/lib/python3.8/asyncio/runners.py", line 43, in run
return loop.run_until_complete(main)

File "/usr/lib/python3.8/asyncio/base_events.py", line 608, in run_until_complete
return future.result()

File "cake_fuzzer.py", line 510, in apply_instrumentation
await inst.apply()

File "/cake_fuzzer/cakefuzzer/instrumentation/instrumentator.py", line 173, in apply
unapplied = await apply(*unapplied)

File "/cake_fuzzer/cakefuzzer/instrumentation/init.py", line 32, in apply
await asyncio.gather(*[p.apply(semaphore) for p in args])

File "/cake_fuzzer/cakefuzzer/instrumentation/override.py", line 35, in apply
raise InstrumentationError(

cakefuzzer.instrumentation.InstrumentationError: Error: Error while instrumenting, got non-zero response from subprocess
Try: php cakefuzzer/phpfiles/instrumentation/rename_function_call.php /var/www/MISP/app/.. header __cakefuzzer_header

PHP Fatal error: Uncaught Error: Call to undefined method PhpParser\ParserFactory::create() in /cake_fuzzer/cakefuzzer/phpfiles/instrumentation/rename_function_call.php:62
Stack trace:
#0 {main}
thrown in /cake_fuzzer/cakefuzzer/phpfiles/instrumentation/rename_function_call.php on line 62
(venv) root@misp:/cake_fuzzer#

i have made the changes in config.ini......but still getting some other error....

Also after successfully completing the execution of all the commands...does the result json contain the vulnerability details pointing towards the exact location of where the error is inside the file?

dawid-czarnecki · 2024-05-04T08:04:53Z

That's weird error. Do you have the php-parser cloned repo directory? If not try to run this script: cakefuzzer/phpfiles/instrumentation/install_php_parser.sh. If yes than it needs deeper investigation.

You don't get the info about the file that contains the vulnerability but the request that will let you to reproduce it.

sreegopcs · 2024-05-04T09:06:08Z

I coudn't fix the issue.....PHP parser is their in the repo....and i even tried using the* install_php_parser.sh* command. Still i got the same error... Can u send me the result.json file of MISP and CEREBRATE..if possible both of them, inorder to examine the result and the way of its analysis

…

On Sat, May 4, 2024 at 1:35 PM Dawid ***@***.***> wrote: That's weird error. Do you have the php-parser cloned repo directory? If not try to run this script: cakefuzzer/phpfiles/instrumentation/install_php_parser.sh. If yes than it needs deeper investigation. You don't get the info about the file that contains the vulnerability but the request that will let you to reproduce it. — Reply to this email directly, view it on GitHub <#16 (comment)>, or unsubscribe <https://github.com/notifications/unsubscribe-auth/AZL5KOMNIKHI4P2BLW5RGKTZASJDXAVCNFSM6AAAAAA3HJIBEKVHI2DSMVQWIX3LMV43OSLTON2WKQ3PNVWWK3TUHMZDAOJUGA3TCNJVGE> . You are receiving this because you were mentioned.Message ID: ***@***.***>

dawid-czarnecki · 2024-05-19T10:31:35Z

Hard to say what's causing the error.
Here is example output with one pre-planted SSRF vulnerability:

[
    {
        "found_in": {
            "strategy_name": "SSRFAttackStrategy",
            "payload": "http://4glkaunm0gijcgqvo1tfp5617sdj19py.00017491695907542774.local",
            "detection_result": "4glkaunm0gijcgqvo1tfp5617sdj19py.00017491695907542774.local",
            "context_location": "",
            "vulnerability_location": {
                "_POST": {
                    "baseurl": "http://4glkaunm0gijcgqvo1tfp5617sdj19py.00017491695907542774.local"
                }
            },
            "vulnerability_id": 0,
            "path": "/TaxiiServers/getRoot/",
            "method": "POST",
            "superglobal": {
                "_GET": {},
                "_POST": {
                    "baseurl": "http://4glkaunm0gijcgqvo1tfp5617sdj19py.00017491695907542774.local",
                    "uri": "/taxii2/"
                },
                "_REQUEST": {},
                "_COOKIE": {
                    "CAKEPHP": "http://4glkaunm0gijcgqvo1tfp5617sdj19py.01588384992370180046.local"
                },
                "_FILES": {},
                "_SERVER": {
                    "HTTP_USER_AGENT": "http://4glkaunm0gijcgqvo1tfp5617sdj19py.01197202265851745442.local",
                    "HTTP_IF_NONE_MATCH": "http://4glkaunm0gijcgqvo1tfp5617sdj19py.03222932666639621773.local",
                    "HTTP_IF_MODIFIED_SINCE": "http://4glkaunm0gijcgqvo1tfp5617sdj19py.01544122491512797937.local",
                    "HTTP_HOST": "127.0.0.1",
                    "HTTP_SEC_FETCH_SITE": "same-origin",
                    "HTTP_ACCEPT": "application/xml"
                }
            }
        }
    }
]

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

source precheck.sh errors #16

source precheck.sh errors #16

kaster-san commented Aug 7, 2023

dawid-czarnecki commented Aug 23, 2023

sreegopcs commented Apr 1, 2024

dawid-czarnecki commented Apr 2, 2024

sreegopcs commented Apr 2, 2024

dawid-czarnecki commented Apr 4, 2024

sreegopcs commented Apr 7, 2024 via email

dawid-czarnecki commented Apr 12, 2024

sreegopcs commented Apr 21, 2024

dawid-czarnecki commented Apr 23, 2024

sreegopcs commented Apr 26, 2024

dawid-czarnecki commented Apr 28, 2024

sreegopcs commented May 1, 2024

dawid-czarnecki commented May 4, 2024

sreegopcs commented May 4, 2024 via email

dawid-czarnecki commented May 19, 2024

source precheck.sh errors #16

source precheck.sh errors #16

Comments

kaster-san commented Aug 7, 2023

dawid-czarnecki commented Aug 23, 2023

sreegopcs commented Apr 1, 2024

dawid-czarnecki commented Apr 2, 2024

sreegopcs commented Apr 2, 2024

dawid-czarnecki commented Apr 4, 2024

sreegopcs commented Apr 7, 2024 via email

dawid-czarnecki commented Apr 12, 2024

sreegopcs commented Apr 21, 2024

dawid-czarnecki commented Apr 23, 2024

sreegopcs commented Apr 26, 2024

dawid-czarnecki commented Apr 28, 2024

sreegopcs commented May 1, 2024

dawid-czarnecki commented May 4, 2024

sreegopcs commented May 4, 2024 via email

dawid-czarnecki commented May 19, 2024