Initial implementation of file accessing using OS Module

[Thirumalai-Shaktivel]

No description provided.

[certik]

Let's get this working first with just one argument. Does this work both with CPython and LPython? Let's get something that works, merge and then improve upon it.

[Thirumalai-Shaktivel]

CPython (Doesn't work):

$ python integration_tests/test_os.py    
Traceback (most recent call last):
  File "/home/thirumalai/Open_Source/lpython/integration_tests/test_os.py", line 14, in <module>
    test()
  File "/home/thirumalai/Open_Source/lpython/integration_tests/test_os.py", line 9, in test
    fd = open(path)
TypeError: open() missing required argument 'flags' (pos 2)

LPython (Works):

$ lpython integration_tests/test_os.py; ./a.out; rm a.out
from ltypes import i64
from os import (open, read, close)

def test():
    path: str 
    path = "in

[Thirumalai-Shaktivel]

Here, when I assign a value 0 to O_RDONLY it throws the following error:

$ lpython integration_tests/test_os.py; ./a.out; rm a.out
Internal Compiler Error: Unhandled exception
Traceback (most recent call last):
  Binary file "/home/thirumalai/Open_Source/lpython/src/bin/lpython", in _start()
  Binary file "/lib/x86_64-linux-gnu/libc.so.6", in __libc_start_main()
  File "/home/thirumalai/Open_Source/lpython/src/bin/lpython.cpp", line 755, in ??
    err = compile_python_to_object_file(arg_file, tmp_o, runtime_library_dir, compiler_options);
  File "/home/thirumalai/Open_Source/lpython/src/bin/lpython.cpp", line 271, in ??
    res = fe.get_llvm3(*asr, diagnostics);
  File "/home/thirumalai/Open_Source/lpython/src/lpython/fortran_evaluator.cpp", line 57, in LFortran::PythonCompiler::get_llvm3(LFortran::ASR::TranslationUnit_t&, LFortran::diag::Diagnostics&)
    run_fn);
  File "/home/thirumalai/Open_Source/lpython/src/libasr/codegen/asr_to_llvm.cpp", line 4170, in LFortran::asr_to_llvm(LFortran::ASR::TranslationUnit_t&, LFortran::diag::Diagnostics&, llvm::LLVMContext&, Allocator&, LFortran::Platform, bool, std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> > const&, std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> > const&)
    v.visit_asr((ASR::asr_t&)asr);
  File "/home/thirumalai/Open_Source/lpython/src/libasr/../libasr/asr.h", line 2395, in LFortran::ASR::BaseVisitor<LFortran::ASRToLLVMVisitor>::visit_asr(LFortran::ASR::asr_t const&)
    void visit_asr(const asr_t &b) { visit_asr_t(b, self()); }
  File "/home/thirumalai/Open_Source/lpython/src/libasr/../libasr/asr.h", line 2373, in ??
    case asrType::unit: { v.visit_unit((const unit_t &)x); return; }
  File "/home/thirumalai/Open_Source/lpython/src/libasr/../libasr/asr.h", line 2396, in LFortran::ASR::BaseVisitor<LFortran::ASRToLLVMVisitor>::visit_unit(LFortran::ASR::unit_t const&)
    void visit_unit(const unit_t &b) { visit_unit_t(b, self()); }
  File "/home/thirumalai/Open_Source/lpython/src/libasr/../libasr/asr.h", line 2231, in ??
    case unitType::TranslationUnit: { v.visit_TranslationUnit((const TranslationUnit_t &)x); return; }
  File "/home/thirumalai/Open_Source/lpython/src/libasr/codegen/asr_to_llvm.cpp", line 911, in LFortran::ASRToLLVMVisitor::visit_TranslationUnit(LFortran::ASR::TranslationUnit_t const&)
    visit_symbol(*item.second);
  File "/home/thirumalai/Open_Source/lpython/src/libasr/../libasr/asr.h", line 2398, in LFortran::ASR::BaseVisitor<LFortran::ASRToLLVMVisitor>::visit_symbol(LFortran::ASR::symbol_t const&)
    void visit_symbol(const symbol_t &b) { visit_symbol_t(b, self()); }
  File "/home/thirumalai/Open_Source/lpython/src/libasr/../libasr/asr.h", line 2241, in ??
    case symbolType::Subroutine: { v.visit_Subroutine((const Subroutine_t &)x); return; }
  File "/home/thirumalai/Open_Source/lpython/src/libasr/codegen/asr_to_llvm.cpp", line 1981, in LFortran::ASRToLLVMVisitor::visit_Subroutine(LFortran::ASR::Subroutine_t const&)
    generate_subroutine(x);
  File "/home/thirumalai/Open_Source/lpython/src/libasr/codegen/asr_to_llvm.cpp", line 2053, in LFortran::ASRToLLVMVisitor::generate_subroutine(LFortran::ASR::Subroutine_t const&)
    this->visit_stmt(*x.m_body[i]);
  File "/home/thirumalai/Open_Source/lpython/src/libasr/../libasr/asr.h", line 2410, in LFortran::ASR::BaseVisitor<LFortran::ASRToLLVMVisitor>::visit_stmt(LFortran::ASR::stmt_t const&)
    void visit_stmt(const stmt_t &b) { visit_stmt_t(b, self()); }
  File "/home/thirumalai/Open_Source/lpython/src/libasr/../libasr/asr.h", line 2259, in ??
    case stmtType::Assignment: { v.visit_Assignment((const Assignment_t &)x); return; }
  File "/home/thirumalai/Open_Source/lpython/src/libasr/codegen/asr_to_llvm.cpp", line 2480, in LFortran::ASRToLLVMVisitor::visit_Assignment(LFortran::ASR::Assignment_t const&)
    this->visit_expr_wrapper(x.m_value, true);
  File "/home/thirumalai/Open_Source/lpython/src/libasr/codegen/asr_to_llvm.cpp", line 2505, in LFortran::ASRToLLVMVisitor::visit_expr_wrapper(LFortran::ASR::expr_t const*, bool)
    this->visit_expr(*x);
  File "/home/thirumalai/Open_Source/lpython/src/libasr/../libasr/asr.h", line 2443, in LFortran::ASR::BaseVisitor<LFortran::ASRToLLVMVisitor>::visit_expr(LFortran::ASR::expr_t const&)
    void visit_expr(const expr_t &b) { visit_expr_t(b, self()); }
  File "/home/thirumalai/Open_Source/lpython/src/libasr/../libasr/asr.h", line 2304, in ??
    case exprType::FunctionCall: { v.visit_FunctionCall((const FunctionCall_t &)x); return; }
  File "/home/thirumalai/Open_Source/lpython/src/libasr/codegen/asr_to_llvm.cpp", line 4070, in LFortran::ASRToLLVMVisitor::visit_FunctionCall(LFortran::ASR::FunctionCall_t const&)
    std::vector<llvm::Value *> args2 = convert_call_args(x, m_name);
AssertFailed: finder != nested_globals.end()

[certik]

That is a real bug in the LLVM backend, the ASR looks good. We'll have to fix it, but let's do it in a separate PR. would you mind please reporting it as an issue?

[Thirumalai-Shaktivel]

But in the backend(LPython), O_RDONLY takes value 0 which makes the test_os.py file work in LPython.
Now, test_os.py works in both LPython and CPython

[namannimmo10]

The integration tests don't pass on my system:

Running: integration_tests/tmp/test_os
from ltypes import i64
from os import (open, read, close)

def test():
    path: str
    path = "in
Running: PYTHONPATH=src/runtime/ltypes python integration_tests/test_os.py
Traceback (most recent call last):
  File "/Users/namannimmo/Applications/oss/lpython/integration_tests/test_os.py", line 14, in <module>
    test()
  File "/Users/namannimmo/Applications/oss/lpython/integration_tests/test_os.py", line 9, in test
    fd = open(path, "r")
TypeError: 'str' object cannot be interpreted as an integer
Command 'PYTHONPATH=src/runtime/ltypes python integration_tests/test_os.py' failed.

[certik]

I see, and it failed at the CI too: https://github.com/lcompilers/lpython/runs/5740034648?check_suite_focus=true#step:11:299, but the CI passed. Let's first fix the CI to fail, and then let's fix the failure.

[certik]

Fixed in #299